Posts Tagged With 'python'

Python 3 on the desktop for Quantal Quetzal

Written by Barry Warsaw in technology on Tue 24 April 2012. Tags: canonical, debian, ubuntu, python, python3,

So, now all the world now knows that my suggested code name for Ubuntu 12.10, Qwazy Quahog, was not chosen by Mark. Oh well, maybe I'll have more luck with Racy Roadrunner.

In any case, Ubuntu 12.04 LTS is to be released any day now so it's time for my semi-annual report on Python plans for Ubuntu. I seem to write about this every cycle, so 12.10 is no exception. We've made some fantastic progress, but now it's time to get serious.

For Ubuntu 12.10, we've made it a release goal to have Python 3 only on the desktop CD images. The usual caveats apply: Python 2.7 isn't going away; it will still probably always be available in the main archive. This release goal also doesn't affect other installation CD images, such as server, or other Ubuntu flavors. The relatively modest goal then only affects packages for the standard desktop CD images, i.e. the alternative installation CD and the live CD.

Update 2012-04-25: To be crystal clear, if you depend on Python 2.7, the only thing that changes for you is that after a fresh install from the desktop CD on a new machine, you'll have to explicitly apt-get install *python2.7. After that, everything else will be the same.

This is ostensibly an effort to port a significant chunk of Ubuntu to Python 3, but it really is a much wider, Python-community driven effort. Ubuntu has its priorities, but I personally want to see a world where Python 3 rules the day, and we can finally start scoffing at Python 2 :).

Still, that leaves us with about 145 binary packages (and many fewer source packages) to port. There are a few categories of packages to consider:

Already ported and available.
This is …

Continue reading »

Debian packaging for Python 2 and 3

Written by Barry Warsaw in technology on Wed 18 January 2012. Tags: python, debian,

Time for another installment of my ongoing mission to convert the world to Python 3! This time, a little Debian packaging-fu for modifying an existing Python 2 package to include support for Python 3 from the same source package.

Today, I added a python3-feedparser package to Ubuntu Precise. What's interesting about this is that, despite various reported problems, upstream feedparser 5.1 claims to support Python 3, via 2to3 conversion. And indeed it does (although the test suite does not).

Before today, Ubuntu had feedparser 5.0.1 in its archive, and while some work has been done to update the Debian package to 5.1, this has not been released. The uninteresting precursor to Python 3 packaging was to upgrade the Ubuntu version of the python-feedparser source package to 5.1. I'll spare you the boring details about missing data files in the upstream tarball, and other problems, since they don't really relate to the Python 3 effort.

The first step was to verify that feedparser 5.1 works with Python 3.2 in a virtualenv, and indeed it does. This is good news because it means that the does the right thing, which is always the best way to start supporting Python 3. I've found that it's much easier to build a solid Debian package if you have a solid in upstream to begin with.

Now, what I'd like to do is to give you a recipe for modifying your existing debian/ directory files to add Python 3 support to a package that already exists for Python 2. This is a little trickier for feedparser because it used an older debhelper standard, and carried some crufty old stuff in its rules file. My first step was to update this to debhelper compatibility level 8 …

Continue reading »

Lessons in porting to Python 3

Written by Barry Warsaw in technology on Wed 07 December 2011. Tags: python, python3, ubuntu,

Yesterday, I completed my port of dbus-python to Python 3, and submitted my patch upstream. While I've yet to hear any feedback from Simon about my patch, I'm fairly confident that it's going in the right direction. This version should allow existing Python 2 applications to run largely unchanged, and minimizes the differences that clients will have to make to use the Python 3 version.

Some of the changes are specific to the dbus-python project, and I included a detailed summary of those changes and my rationale behind them. There are lots of good lessons learned during this porting exercise that I want to share with you, have a discussion about, and see if there aren't things we core Python developers can do in Python 3.3 to make it even easier to migrate to Python 3.

First, some background. D-Bus is a project for same-system interprocess communication, and it's an essential component of any Linux desktop. The D-Bus system and C API are mature and well-defined, and there are bindings available for many programming language, Python included of course. The existing dbus-python package is only compatible with Python 2, and most recommendations are to use the Gnome version of Python bindings should you want to use D-Bus with Python 3. For us in Ubuntu, this isn't acceptable though because we must have a solution that supports KDE and potentially even non-UI based D-Bus Python servers. Several ports of dbus-python to Python 3 have been attempted in the past, but none have been accepted upstream, so naturally I took it as a challenge to work on a new version of the port. After some discussion with the upstream maintainer Simon McVittie, I had a few requirements in mind:

  • One code base for both Python 2 and Python 3 …

Continue reading »

An update on Ubuntu's Python plans

Written by Barry Warsaw in technology on Wed 23 November 2011. Tags: python, ubuntu,

Earlier this month, I attended UDS-P (the Ubuntu Developers Summit for 12.04 Precise Pangolin). 12.04 is an LTS release, or Long Term Support, meaning it will be officially supported on both the desktop and server for five years.

During the summit we reiterated our plans for Python on Ubuntu, both for 12.04 LTS, and our vision of Python two years from now for the next LTS, 14.04. I'm here to provide an update from my last report, which was written after UDS-O for 11.10.

As per those previous plans, we've removed Python 2.6 from Ubuntu 12.04. Now you have only Python 2.7 and 3.2. Dropping Python 2.6 may cause some inconvenience for data centers which like to upgrade only between LTS's but don't want to have to upgrade both their operation system and their Python version. Canonical services such as Launchpad and Landscape fall into this camp. The biggest problem is that Python 2.7 is not available for the last LTS, i.e. 10.04. To mitigate this, we decided to create a PPA containing Python 2.7 and a bunch of packages that services such as Launchpad will need to do their porting. This now exists, although it hasn't yet been tested with any development branches of Launchpad or Landscape as far as I'm aware. If you have your own data center porting task ahead of you, you can also use this PPA, and if there are additional packages you need for 10.04, you can create your own PPA which depends on ours, and build those extra packages there.

But that's all boring stuff. Let's have some fun!

The other decision we made concerns Python 3. I strongly feel that the Python community is very close to …

Continue reading »

PEP 382 sprint summary

Written by Barry Warsaw in technology on Wed 22 June 2011. Tags: debian, packaging, python, ubuntu,

So, yesterday (June 21, 2011), six talented and motivated Python hackers from the Washington DC area met at Panera Bread in downtown Silver Spring, Maryland to sprint on PEP 382. This is a Python Enhancement Proposal to introduce a better way for handling namespace packages, and our intent is to get this feature landed in Python 3.3. Here then is a summary, from my own spotty notes and memory, of how the sprint went.

First, just a brief outline of what the PEP does. For more details please read the PEP itself, or join the newly resurrected import-sig for more discussions. The PEP has two main purposes. First, it fixes the problem of which package owns a namespace's file, e.g. zope/ for all the Zope packages. In essence, it eliminate the need for these by introducing a new variant of .pth files to define a namespace package. Thus, the zope.interfaces package would own zope/zope-interfaces.pth and the zope.components package would own zope/zope-components.pth. The presence of either .pth file is enough to define the namespace package. There's no ambiguity or collision with these files the way there is for zope/ This aspect will be very beneficial for Debian and Ubuntu.

Second, the PEP defines the one official way of defining namespace packages, rather than the multitude of ad-hoc ways currently in use. With the pre-PEP 382 way, it was easy to get the details subtly wrong, and unless all subpackages cooperated correctly, the packages would be broken. Now, all you do is put a * in the .pth file and you're done.

Sounds easy, right? Well, Python's import machinery is pretty complex, and there are actually two parallel implementations of it in Python 3.3, so gaining traction on …

Continue reading »

Python plans for Ubuntu 11.10 and 12.04

Written by Barry Warsaw in technology on Thu 19 May 2011. Tags: canonical, python, ubuntu, uds-o,

TL;DR: Ubuntu 12.04 LTS will contain only Python 2.7 and 3.2, while Ubuntu 11.10 will contain Python 3.2, 2.7 and possibly 2.6, but possibly not.

Last week, I attended the Ubuntu Developer Summit in Budapest, Hungary. These semi-annual events are open to everyone, and hundreds of people participate both in person and remotely. Budapest's was called UDS-O, where the 'O' stands for Oneiric Ocelot, the code name for Ubuntu 11.10, which will be released in October 2011. This is where we did the majority of planning for what changes, new features, and other developments you'll find in the next version of Ubuntu. UDS-P will be held at the end of the year in Orlando, Florida and will cover the as yet unnamed 12.04 release, which will be a Long Term Support release.

LTS releases are special, because we make longer guarantees for official support: 3 years on the desktop and 5 years on the server. Because of this, we're making decisions now to ensure that 12.04 LTS is a stable, confident platform for years to come.

I attended many sessions, and there is a lot of exciting stuff coming, but I want to talk in some detail about one area that I'm deeply involved in. What's going to happen with Python for Oneiric and 12.04 LTS?

First, a brief summary of where we are today. Natty Narwhal is the code name for Ubuntu 11.04, which was released back in April and is the most recent stable release. It is not an LTS though; the last LTS was Ubuntu 10.04 Lucid Lynx, release back in October 2010. In Lucid, the default Python (i.e. /usr/bin/python) is 2.6 and Python 2.7 is not officially …

Continue reading »

Release that rewrite on 11.11.11

Written by Barry Warsaw in technology on Mon 04 April 2011. Tags: floss, mailman, opensource, python,

I know that the Mailman 3 project is not alone in procrastinating getting out a release of its major rewrite. It's hard work to finish a rewrite on your own copious spare time. I was just chatting with Thomas Waldmann of the Moin project on IRC, and he lamented a similar story about the Moin 2 release. Then he said something that really made me sit up straight:

<ThomasWaldmann> 11.11.11 would be a great date for something :)

Yes, it would! We have the 2011 Google Summer of Code happening soon (students, you have until April 8th to submit your applications) so many free and open source software projects will get some great code coming soon. And November is far enough out that we can plan exactly what a "release" means. Here's what I propose:

Let's make November 11, 2011 the "Great FLOSS Release Day". If you're working on an open source project undergoing a major new version rewrite, plan on doing your release on 11.11.11. It can be a beta or final release, but get off your butts and make it happen! There's nothing like a good deadline to motivate me, so Mailman 3 will be there. Add a comment here if you want your project to be part of the event!

Charming Snakes and Shaving Yaks

Written by Barry Warsaw in technology on Mon 28 March 2011. Tags: debugging, python, ubuntu,

For the last couple of days I've been debugging a fun problem in the Ubuntu tool called Jockey. Jockey is a tool for managing device drivers on Ubuntu. It actually contains both a command-line and a graphical front-end, and a dbus backend service that does all the work (with proper authentication, since it modifies your system). None of that is terribly relevant to the problem, although the dbus bit will come back to haunt us later.

What is important is that Jockey is a Python application, written using many Python modules interfacing to low-level tools such as apt and dbus. The original bug report was mighty confusing. Aside from not being reproducible by myself and others, the actual exception made no fricken sense! Basically, it was code like this that was throwing a TypeError:

_actions = []
# _actions gets appended to at various times and later...
for item in _actions[:]:
    # do something

Everyone who reported the problem said the TypeError was getting thrown on the for-statement line. The exception message indicated that Python was getting some object that it was trying to convert to an integer, but was failing. How could you possible get that exception when either making a copy of a list or iterating over that copy? Was the list corrupted? Was it not actually a list but some list-like object that was somehow returning non-integers for its min and max indexes?

To make matters worse, this little code snippet was in Python's standard library, in the subprocess module. A quick search of Python's bug database did reveal some recent threads about changes here, made to ensure that popen objects got properly cleaned up by the garbage collector if they weren't cleaned up explicitly by the program. Note that we're using Python 2.7 here, and after some reading …

Continue reading »

What we do

Written by Barry Warsaw in technology on Thu 09 September 2010. Tags: canonical, python, ubuntu,

My friends and family often ask me what I do at my job. It's easy to understand when my one brother says he's a tax accountant, but not so easy to explain the complex world of open source software development I live in. Sometimes I say something to the effect: well, you know what Windows is, and you know what the Mac is right? We're building a third alternative called Ubuntu that is free, Linux-based and in most cases, much better. Mention that you won't get viruses and it can easily breathe new life into that old slow PC you shudder to turn on, and people at least nod their heads enthusiastically, even if they don't fully get it.

I've been incredibly fortunate in my professional career, to have been able to share the software I write with the world for almost 30 years. I started working for a very cool research lab with the US Federal government while still in high school. We had a UUCP connection and were on the early Arpanet, and because we were funded by the US taxpayer, our software was not subject to copyright. This meant that we could share our code with other people on Usenet and elsewhere, collaborate with them, accept their suggestions and improvements, and hopefully make their lives a little better, just as others around the world did for us. It was free and open source software before such terms were coined.

I've never had a "real job" in the sense of slaving away in a windowless cube writing solely proprietary software that would never see the light of day. Even the closed source shops I've worked at have been invested somehow in free software, and with varying degrees of persuasion, have both benefited from and contributed to the …

Continue reading »

import this and the Zen of Python

Written by Barry Warsaw in technology on Tue 08 June 2010. Tags: python, history,

Richard Jones is working on a talk for PyCon Australia and asked me about the history of the Zen of Python, Tim Peters' eternal words of wisdom, often quoted, on the essential truths of Python. At first, I couldn't find a reference to the first publication of this list, but then I did a better search of my archives and found that it was originally sent to the python-list mailing list on June 4, 1999, under the subject "The Python Way".

Interestingly enough, because I couldn't find that first reference immediately, I went back into my archives and researched the "this" module. Did you know that if you type the following at a modern Python interpreter, you get the Zen of Python?

% python3 -c "import this"
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

The story behind "import this" is kind of funny, and occurred totally behind the scenes, so I thought it might be interesting to relate how it happened …

Continue reading »