January 18, 2007

Python 3.0 Alarmism: Pay No Attention

I don't follow the python-dev list religiously, but I do like to keep my eye on (what seem to me) the more important threads. Recently there's been a "Proposed 3.0 compatibilty module" thread in which various luminaries have discussed how migration to the new version might be made easier for current users. The general conclusion, now the dust has settled, seems to be that no compromise is to be made on the 3.0 design, that it will clearly break compatibility with 2.5 (no surprise there), and that 2.6 should contain enough 3.0 features that onward migration with the help of an automated translation tool should not be a problem.

In an earlier thread on the proposed bytes type, I expressed to Guido my feeling that it's important the development of 3.0 doesn't fragment the community in the way suggested by theand asked him whether there were plans to discuss 3.0 with implementation teams from the versions other than CPython.

Yesterday there were some rather alarmist comments by Calvin Spealman, suggesting it's at least possible that the sky is falling and that forks are coming. One quote from Spealman's blog is revealing: "What few ideas I do have towards avoiding the problems are not well thought out, probably are missing vital information, or are simply stupid." You said it, Calvin, and I'm not convinced that your approach was the most helpful one you could have taken. Fortunately our technical blogs don't usually make the front pages of the Washington Times - I have been known to suffer from foot-in-mouth myself.

Now I see that Chas Whitacre has also read Spealman's blog and is looking for advice in whether to adopt IronPython. He says "So what would really help me would be posts demonstrating the respective market shares of the various Python implementations, and resources detailing the differences between them". This is a reasonable request, but I have a nasty feeling that these two blog posts between them will spark alarm, that those with an axe to grind will jump in gleefully, and that a shitstorm will result.

Guido replied off-list, so all I can say is that he has confirmed that the upcoming PyCon represents a great opportunity to address these issues. In fact, though, the situation today is already somewhat fragmented. Jython support appears to lag at least a release-and-a-half behind and IronPython and PyPy are currently (as far as I know) still targeting 2.4. Just the same they are all implementing Python, and that's the important fact to keep in mind.

I believe it's necessary to try to ensure that all implementations are encouraged to stay in step with the reference implementation (which at the moment remains CPython). Even if this proves difficult and they don't all make it to the promised land of 3.0 their users will still be better off running 2.3, or 2.4, or 2.5 than using other, less useful, languages. Microsoft managed to break compatibility on Visual Basic without the world coming to an end, and the user community involved there was enormous, so let's not pretend that this is any more than a storm in a teacup.

Of course the information that Chad Whitacre asked for would be both interesting and useful, so if anyone has information to share (preferably facts rather than opinions, but I suspect few can be discovered) I would love to see it posted, here or elsewhere. One fact I can give you is that Python's popularity continues to grow. As Python is now integrated into most popular Linux/*nix distributions (and indeed scripts many of their installation routines) a much larger percentage of the downloads are for Windows than was the case a year or two ago. H-P now installs Python on their Windows PCs to help script their installs, so the movement towards world domination continues. You can even run Python on the Nokia 700 web tablet. This is a fundamentally healthy situation.


Anonymous said...

Thanks for reading, Steve! I agree that alarmism only goes so far, and that the "excited" camp will do best to follow up with some good, calm info.

Doug Napoleone said...

As any language, operating system, or complex framework grows, two things happen. First, the project will reach a point where it needs to break from the past in order to make progress into the future. This may happen many times over the life of the project. The second is the project will grow beyond its existing ecosystem. Again this can happen many times over the projects life.

When these two things happen at relatively the same time, there always seems to be this type of strife. I have seen it happen in countless open source projects, and many many closed source products. In all but one case (Fedora, which had other outside factors), did the project make a drastic Fork. Both Ruby and Perl are dealing with the same issues with new interpreter and language spec respectively.

It is important to note that this is the second break from the past that python has made. I am always amazed at the places Python 1.5 keeps popping up. The 1.5 spec is solid, stable, and still popular. Some researchers where I work still use is and scoff at 'all that fancy stuff'.

As the new larger python ecosystem becomes aclimated the fears will fade away. The aclimation will most likely include a better exposed development process, better understanding of that process, and some changes to that process. This is just what happens when more people with different views and requirements want to be involved. The re-commitment to timley python-dev summaries, summaries of other development forums, migration to svn, and the new bug tracking system are all evidence of this. It just means python is a very healthy language.

I am a little suprized there is concern over IronPython given Microsoft's aparent commitment to the effort, it's very active development and Jim Hugunin's involvement in general python development. As for migrating to Python3.0 Jeremy Hylton will be giving a talk on a
transformation tool

for just that purpose, partially based on the AST.

Looking over the list of talks, all the python implementations that I know of are well represented:

I am assuming your cryptic comment about Guido and PyCon are about his Python3000 keynote and the Pyhon-Dev panel. PyCon doesn't feel like an opertunity to resolve the issues, as much as the resolutions, the sharing of knowledge and understanding, being the inevitable end result. No one speech or panel will do the trick, but the combined message from all the presentations is overwhelming. Everyone is looking forward to Python 3.0; no one is waiting, holding off development, or forking. People are doing amazing, amazing things now, with ambitious plans for the future.

Ok, that was much more than I intended to write, it just pressed a button I guess.

Steve said...

My comments about Guido might have appeared cryptic when they weren't intended to be. It's simply that the reply I got wasn't addressed to a public list, and didn't contain much information anyway.

Guido is quite capable of representing his own views and I don't want to put words in his mouth. By "address these issues" I simply meant "meet with the people involved in the other ports and get their views on migration to Python 3.0". Sorry if it sounded like more than that: I expect Guido's presentations to be technical, not political.

There is, however, a Python Developers' Panel at PyCon where it looks like these issues might be discussed. I have an idea they will also be the topic of conversation over many beers and meals.