Posts Tagged With 'python3'
Background
Snappy Ubuntu Core is a new edition of the Ubuntu you know and love, with
some interesting new features, including atomic, transactional updates, and a
much more lightweight application deployment story than traditional
Debian/Ubuntu packaging. Much of this work grew out of our development of a
mobile/touch based version of Ubuntu for phones and tablets, but now Ubuntu
Core is available for clouds and devices.
I find the transactional nature of upgrades to be very interesting. While you
still get a perfectly normal Ubuntu system, your root file system is
read-only, so traditional apt-get based upgrades don't work. Instead, your
system version is image based; today you are running image 231 and tomorrow
a new image is released to get you to 232. When you upgrade to the new image,
you get all the system changes. We support both full and delta upgrades
(the latter which reduces bandwidth), and even phased updates so that we can
roll out new upgrades and quickly pull them from the server side if we notice
a problem. Snappy devices even support rolling back upgrades on a single
device, by using a dual-partition root file system. Phones generally don't
support this due to lack of available space on the device.
Of course, the other part really interesting thing about Snappy is the
lightweight, flexible approach to deploying applications. I still remember my
early days learning how to package software for Debian and Ubuntu, and now
that I'm both an Ubuntu Core Developer and Debian Developer, I understand
pretty well how to properly package things. There's still plenty of black art
involved, even for relatively easy upstream packages such as
distutils/setuptools-based Python packages available on the Cheeseshop (er,
PyPI). The Snappy approach on Ubuntu Core is much more lightweight and easy,
and …
Continue reading »
I'm writing a bunch of new code these days for Ubuntu Touch's Image Based
Upgrade system. Think of it essentially as Ubuntu Touch's version of
upgrading the phone/tablet (affectionately called phablet) operating system
in a bulk way rather than piecemeal apt-get s the way you do it on a
traditional Ubuntu desktop or server. One of the key differences is that a
phone has to detour through a reboot in order to apply an upgrade since its
Ubuntu root file system is mounted read-only during the user session.
Anyway, those details aren't the focus of this article. Instead, just realize
that because it's a pile of new code, and because we want to rid ourselves of
Python 2, at least on the phablet image if not everywhere else in Ubuntu, I
am prototyping all this in Python 3, and specifically 3.3. This means that
I can use all the latest and greatest cool stuff in the most recent stable
Python release. And man, is there a lot of cool stuff!
One module in particular that I'm especially fond of is contextlib. Context
managers are objects implementing the protocol behind the with
statement, and they are typically used to guarantee that some resource is
cleaned up properly, even in the event of error conditions. When you see code
like this:
with open(somefile) as fp:
data = fp.read()
you are invoking a context manager. Python was clever enough to make file
objects support the context manager protocol so that you never have to
explicitly close the file; that happens automatically when the with
statement completes, regardless of whether the code inside the with
statement succeeds or raises an exception.
It's also very easy to define your own context managers to properly handle
other kinds of resources. I won't go …
Continue reading »
There's a lot of Python nostalgia going around today, from Brett Cannon's 10
year anniversary of becoming a core developer, to Guido reminding us that
he came to the USA 18 years ago. Despite my stolen time machine keys, I
don't want to dwell in the past, except to say that I echo much of what Brett
says. I had no idea how life changing it would be -- on both a personal and
professional level -- when Roger Masse and I met Guido at NIST at the
first Python workshop back in November 1994. The lyric goes: what a long
strange trip it's been, and that's for sure. There were about 20 people
at that first workshop, and 2500 at Pycon 2013.
And Python continues to hold little surprises. Just today, I solved a bug in
an Ubuntu package that's been perplexing us for weeks. I'd looked at the code
dozens of times and saw nothing wrong. I even knew about the underlying
corner of the language, but didn't put them together until just now. Here's a
boiled down example, see if you can spot the bug!
import sys
def bar(i):
if i == 1:
raise KeyError(1)
if i == 2:
raise ValueError(2)
def bad():
e = None
try:
bar(int(sys.argv[1]))
except KeyError as e:
print('ke')
except ValueError as e:
print('ve')
print(e)
bad()
Here's a hint: this works under Python 2, but gives you an
UnboundLocalError on the e variable under Python 3.
Why?
The reason is that in Python 3, the targets of except clauses are del'd from
the current namespace after the try...except clause executes. This is
to prevent circular references that occur when the exception is bound to the
target. What is surprising and non-obvious is that the name is deleted …
Continue reading »
For UDS-R for Raring (i.e. Ubuntu 13.04) in Copenhagen, I sponsored
three blueprints. These blueprints represent most of the work I will be doing
for the next 6 months, as we're well on our way to the next LTS, Ubuntu 14.04.
I'll provide some updates to the other blueprints later, but for now, I want
to talk about OAuth and Python 3. OAuth is a protocol which allows you to
programmatically interact with certain website APIs, in an authenticated
manner, without having to provide your website password. Essentially, it
allows you to generate an authorization token which you can use instead, and
it allows you to manage and share these tokens with applications, so that you
can revoke them if you want, or decide how and which applications to trust to
act on your behalf.
A good example of a site that uses OAuth is Launchpad, but many other sites
also support OAuth, such as Twitter and Facebook.
There are actually two versions of OAuth out there. OAuth version 1 is
definitely the more prevalent, since it has been around for years, is
relatively simple (at least on the client side), and enshrined in RFC 5849.
There are tons of libraries available that support OAuth v1, in a multitude
of languages, with Python being no exception.
OAuth v2 is much less common, since it is currently only a draft
specification, and has had its share of design-by-committee controversy.
Still, some sites such as Facebook do require OAuth v2.
One of the very earliest Python libraries to support OAuth v1, on both the
client and server side, was python-oauth (I'll use the Debian package names
in this post), and on the Ubuntu desktop, you'll find lots of scripts and
libraries that use python-oauth. There are major problems with …
Continue reading »
Recently, as part of our push to ship only Python 3 on the Ubuntu 12.10
desktop, I've helped several projects update their internationalization
(i18n) support. I've seen lots of instances of suboptimal Python 2 i18n code,
which leads to liberal sprinkling of cargo culted .decode() and
.encode() calls simply to avoid the dreaded UnicodeError s. These get
worse when the application or library is ported to Python 3 because then even
the workarounds aren't enough to prevent nasty failures in non-ASCII
environments (i.e. the non-English speaking world majority :).
Let's be honest though, the problem is not because these developers are crappy
coders! In fact, far from it, the folks I've talked with are really really
smart, experienced Pythonistas. The fundamental problem is Python 2's 8-bit
string type which doubles as a bytes type, and the terrible API of the
built-in Python 2 gettext module, which does its utmost to sabotage your
Python 2 i18n programs. I take considerable blame for the latter, since I
wrote the original version of that module. At the time, I really didn't
understand unicodes (this is probably also evident in the mess I made of the
email package). Oh, to really have access to Guido's time machine.
The good news is that we now know how to do i18n right, especially in a
bilingual Python 2/3 world, and the Python 3 gettext module fixes the most
egregious problems in the Python 2 version. Hopefully this article does some
measure of making up for my past sins.
Stop right here and go watch Ned Batchelder's talk from PyCon 2012 entitled
Pragmatic Unicode, or How Do I Stop the Pain? It's the single best
description of the background and effective use of Unicode in Python you'll
ever see. Ned does a brilliant job of …
Continue reading »
So, now all the world now knows that my suggested code name for Ubuntu 12.10,
Qwazy Quahog, was not chosen by Mark. Oh well, maybe I'll have more luck
with Racy Roadrunner.
In any case, Ubuntu 12.04 LTS is to be released any day now so it's time
for my semi-annual report on Python plans for Ubuntu. I seem to write about
this every cycle, so 12.10 is no exception. We've made some fantastic
progress, but now it's time to get serious.
For Ubuntu 12.10, we've made it a release goal to have Python 3 only on the
desktop CD images. The usual caveats apply: Python 2.7 isn't going away; it
will still probably always be available in the main archive. This release
goal also doesn't affect other installation CD images, such as server, or
other Ubuntu flavors. The relatively modest goal then only affects
packages for the standard desktop CD images, i.e. the alternative installation
CD and the live CD.
Update 2012-04-25: To be crystal clear, if you depend on Python 2.7, the
only thing that changes for you is that after a fresh install from the
desktop CD on a new machine, you'll have to explicitly apt-get install
*python2.7. After that, everything else will be the same.
This is ostensibly an effort to port a significant chunk of Ubuntu to Python
3, but it really is a much wider, Python-community driven effort. Ubuntu has
its priorities, but I personally want to see a world where Python 3 rules the
day, and we can finally start scoffing at Python 2 :).
Still, that leaves us with about 145 binary packages (and many fewer source
packages) to port. There are a few categories of packages to consider:
- Already ported and available.
- This is …
Continue reading »
My last post on Python 3 porting got some really great responses, and I've
learned a lot from the feedback I've seen. I'm here to rather briefly outline
a few additional tips and tricks that folks have sent me and that I've learned
by doing other ports since then. Please keep them coming, either in the blog
comments or to me via email. Or better yet, blog about your experiences
yourself and I'll link to them from here.
One of the big lessons I'm trying to adopt is to support Python 3 in
pure-Python code with a single code base. Specifically, I'm trying to avoid
using 2to3 as much as possible. While I think 2to3 is an excellent tool that
can make it easier to get started supporting both Python 2 and Python 3 from a
single branch of code, it does have some disadvantages. The biggest problem
with 2to3 is that it's slow; it can take a long time to slog through your
Python code, which can be a significant impediment to your development
velocity. Another 2to3 problem is that it doesn't always play nicely with
other development tools, such as python setup.py test and virtualenv, and
you occasionally have to write additional custom fixers for conversion that
2to3 doesn't handle.
Given that almost all the code I'm writing these days targets Python 2.6 as
the minimal supported Python 2 version, 2to3 may just be unnecessary. With my
dbus-python port to Python 3, and with my own flufl packages, I'm
experimenting with ignoring 2to3 and trying to write one code base for all of
Python 2.6, 2.7, and 3.2. My colleague Michael Foord has been pretty
successful with this approach going back all the way to Python 2.4, so 2.6 as
a …
Continue reading »
Yesterday, I completed my port of dbus-python to Python 3, and submitted
my patch upstream. While I've yet to hear any feedback from Simon about my
patch, I'm fairly confident that it's going in the right direction. This
version should allow existing Python 2 applications to run largely unchanged,
and minimizes the differences that clients will have to make to use the Python
3 version.
Some of the changes are specific to the dbus-python project, and I included
a detailed summary of those changes and my rationale behind them. There
are lots of good lessons learned during this porting exercise that I want to
share with you, have a discussion about, and see if there aren't things we
core Python developers can do in Python 3.3 to make it even easier to migrate
to Python 3.
First, some background. D-Bus is a freedesktop.org project for same-system
interprocess communication, and it's an essential component of any Linux
desktop. The D-Bus system and C API are mature and well-defined, and there
are bindings available for many programming language, Python included of
course. The existing dbus-python package is only compatible with Python 2,
and most recommendations are to use the Gnome version of Python bindings
should you want to use D-Bus with Python 3. For us in Ubuntu, this isn't
acceptable though because we must have a solution that supports KDE and
potentially even non-UI based D-Bus Python servers. Several ports of
dbus-python to Python 3 have been attempted in the past, but none have been
accepted upstream, so naturally I took it as a challenge to work on a new
version of the port. After some discussion with the upstream maintainer Simon
McVittie, I had a few requirements in mind:
- One code base for both Python 2 and Python 3 …
Continue reading »