State of the Python Universe

Since many people will be commenting on this presentation, I'll see if I can't just try to summarise for those who want the headlines without having to read the whole story.

The Move to Subversion - the main reason for the move was the slowness of SourceForge. [Apparently Guido got a message yesterday to say that SF do now have Subversion available]. I wonder how many projects that didn't have the resources to move to their own SVN repository are still struggling with delays before their commits appear in the anonymously-available versions. Maybe SVN on SF has helped.

Buildbot - with the introduction of this system we expect to see big improvements in the detection of single-platform errors and checkins performed without passing tests.

The Cheese Shop - has been a great step forward in Python non-core code availability. More people should be using it to release their code.

www.python.org - would you like to be a python.orb webmaster? We are looking for helpers, and the new design was specifically intended to make content management easier. Email to pydotorg@python.org if you'd like to help.

Python 2.5 release schedule - expect to see 2.5 around September 30. There will be a 2.4.3 bugfix release before the first alpha of the new release, and a final 2.4.4 (barring real security problems) after the 2.5 dust has cleared.

The import mechanism will change, introducing a mechanism to specify import from a position relative to the current package.

The new conditional expressions have a slightly quirky syntax, and it sounds as though Guido has made this choice to discourage its over-use. The essence of his remarks was that it's not Pythonic (though he didn't use that word) to write too many lengthy expressions spanning multiple lines.

It will become possible to write exception handling constructs with try ... except ... finally rather than having two nested constructs. One of the advantages of Guido's time spent in Java-jail is that he realised after reading the Java documentation that this does not introduce any ambiguity.

Generator semantics will change to allow the yield keyword to return a value rather than introducing a statement. This value is returned by calling the generator's .send() method. It's actually possible to cause the yield to raise an exception by calling the generator's .throw() method. Generators also have a .close() method that raises a GeneratorExit exception. This new method will be closed when the generator is garbage-collected.

The with statement is intended to ease the coding of critical sections which require both initialization and clean-up. with is followed by an expression whose value's __context__() method returns a value that has .__enter__() and .__exit__() methods. The expression is known as the context manager and in the majority of cases __context__() can return self, allowing all three methods to be defined in a single class. Some context managers do need to be independent, however, which is why the semantics were defined this way. Certain standard objects will be redefined as context managers to allow their use in with statements, the major ones being mutexes and lock, decimal objects and files.

Exceptions are finally going to become new-style classes, with the inheritance tree rooted at the new BaseException. The end of string exceptions is now in sight. This may eventually lead to a bare except clause becoming illegal (sensible, because this forces you to use an interface in the sys module to access the exception). User-defined exceptions will continue to use Exception as their base class, but KeyboardInterrupt and SystemExit will no longer be subclasses of Exception, allowing cleaner handling of the mainstream exceptions.

Types will be able to define an __index__() method, guaranteed to return an integer or long value. This has been introduced principally to meet the requirements for slicing of complex structures, where the __int__() method isn't appropriate.

The compiler has been revised to create an abstract syntax tree rather than a concrete one. This was a development that almost didn't make it, after much work. Only after Guido said "let's forget it" did the developers realise that they didn't want to lose results of the hard work and finally integrate it into the trunk. This has led to improvements in the maintainability of the compiler, allowing developers to make experimental syntax changes more easily.

Other highlights - Pretty much invisible to users, but invaluable for C developers on 64-bit machines, is the new ssize_t type. This will allow strings (and even lists!) to be longer than 2G elements. The -m command-line option will now allow you to run package submodules as well as top-level modules. This has already proved its value in running regression tests.

Python eggs support is at least on the horizon, making simpler installation a practical possibility for the average Python user.

A new dictionary object, collections.defaultdict, will allow much more flexibility and efficiency for certain dictionary applications.

lambda will survive beyond Python 2.x because "it's perfect" (loud applause). Because it's perfect, no changes will be made.

Q: Will Python access to the AST allow, for example, the implementation of LISP-style macros?
A: I suppose it could be abused for that. (AMK, after galloping to the microphone: The AST BoF session will be discussing this topic at 3:15)

Q: Was "exception filtering" considered?
A: No

Q: Can you add a __missing__ method to a dict?
A: No, builtin types can't be modified in that way.

Q: What happens when two generators' finally clauses call each other?
A: I have no idea. There's a fair chance that it will work, but pathological cases can doubtless be constructed. Don't try and play games with this: you won't even understand what it means yourself!

Q: Can a defaultdict's factory function access the key value?
A: No. To pass the key to the factory function would slow things down. You'll have to override __missing__ in a subclass to do this, as the standard implementation is focused on the common use cases.

Q: How do you like working at Google?
A: Ask me during the break. I love it!