While I am on the subject of conference organization, I don't believe I have recorded what a terrific job Andrew Kuchling has done in putting PyCon 2006 together. I know from personal experience that the organization of this conference places huge stresses on the chairman, but Andrew has performed magnificently.
This PyCon has been better in so many respects than the three that preceded it. I also know from the chatter I've seen in the background during the preceding year that things are becoming rather better codified to assist future organizers. PyCon will continue to improve.
Well done, Andrew!
"Any sufficiently advanced technology is indistinguishable from magic" -- Arthur C Clarke
Musings from the technological frontier
February 28, 2006
Wirelessless
Further to my original remarks about the network at PyCon, this has probably been the biggest complaint I've heard people make, and the post-mortem continues as wireless associations continue to fade in and out during the sprints. Considering that
a) The networking company is at least part-owned by Marriott, and
b) A five-figure sum was negotiated for the wireless coverage and Internet connectivity
the results were very disappointing. For the record STSN didn't have anyone on-site, and there was no evidence that they had sent anyone over beforehand to do any testing and verify that the network coverage would be robust when descended upon by a crowd of geeks, most of whom were expecting the same kind of coverage we got at GWU.
I've no wish to prejudice any negotiations here, so I'll content myself with saying that during the sprint startups, when we shared a room with the organizers' debriefing, I was reminded how difficult it is to get everything right when you depend on third parties for so much of it. This issue isn't over yet, and meetings will follow to pick over the carcass.
a) The networking company is at least part-owned by Marriott, and
b) A five-figure sum was negotiated for the wireless coverage and Internet connectivity
the results were very disappointing. For the record STSN didn't have anyone on-site, and there was no evidence that they had sent anyone over beforehand to do any testing and verify that the network coverage would be robust when descended upon by a crowd of geeks, most of whom were expecting the same kind of coverage we got at GWU.
I've no wish to prejudice any negotiations here, so I'll content myself with saying that during the sprint startups, when we shared a room with the organizers' debriefing, I was reminded how difficult it is to get everything right when you depend on third parties for so much of it. This issue isn't over yet, and meetings will follow to pick over the carcass.
On Planet Python?
Someone yesterday posted that this was "the most underground blog of 2006". I foolishly hadn't thought much about how people might get to it, since like most blogs it's not intended as much more than a place to record random thoughts and impressions. Ephemeral is the word I was looking for (and no jokes about "any port in a storm").
Well, according to Andrew Kuchling this blog should now appear on Planet Python, so I'm making this posting just to see. That might get it into Google somehow.
Well, according to Andrew Kuchling this blog should now appear on Planet Python, so I'm making this posting just to see. That might get it into Google somehow.
February 25, 2006
Lightning Talks (1)
High-Speed Stuff, No Corrections
This is a stream-of-consciousness dump of the lightning talk sessions.
Big and Bold Slides with S5 David Goodger: With Python and docutils you can do large-slide presentations. Q: Can you print your slides? A: Yes, but the layout needs work. CSS anyone? Q: Can you include images? A: Yes, of all kinds? Q: Does it work with IE? A: Yes.
Testing Web Apps with Twill and ??? Titus ??? Twill lets you browse to web pages and fill out forms. Very engaging interactive demonstration. Nice humor: "...and of course the great thing about Python is there are so many different web frameworks you can test". I think I'm going to try twill. Q: Will that work with mod_python? A: It works client side, not server side.
Building MerchantCircle.com David Creemer: They are building a CheeryPy-based online marketing tool for small local merchants. Lots of Python in the application, so thanks. Found it bery good to develop in the production environment. Found FormEncode, Routes, PostgreSQL and Tsearch2 very useful. Avoid magic, be(come) a DBA, hire the best and never stop cleaning. Q: Did you use embedded Python stored procedures? A: We've played with it, but no. Q: Would you like something similar to routes in CherryPy? A: I'd love to see that.
Python Argetina Facundo Batista: We are a user group, and have a portal web site, recently migrated away from Plone to MoinMoin. We have developed a stable workshop in a national university, and given courses. We have T-shirts for sale. We have meetings, talk a lot and drink a lot! We are planning a national meeting to allow people from the different provinces to meet each other, and want to organize a job board, talks in various cities, and keep reaching out to Argentinian users. Q: How small are your smallest T-shirts? A: Medium, but htat's smaller that your medium! (laughter). We don't have XX (more laughter). Q: What other languages are the universities teaching? A: C++ and Java.
Chandler ??? Chandler handles calendar events now, shortll appointments and to-do lists, eventually email. It also lets you share information via a WebDAV repository. 0.6.1 is the latest release, and the speaker is using it live although there arae some rough edges. Built on wxPython and twisted. The GUI allows multiple calendars to be overlaid, which is neat. An event can occur in multiple calendars. Eventually they want Chandler to incorporate many extensions from the Python community. Q: License? A: It's an open source project, with an open license.
How to Replace Yourself with a Small Bot Ben Collins-Sussman After five years of daily IRC use the speaker started to use ircbot.py and generate bots of various kinds, starting with pinkybot and wolfbot. He moved to work with Google, and realised that a bot could take his place in the various communities, and sussbot was born. It does nothing except respond with useless platitudes such as "have you checked the FAQ?" little happened, but some users did undertake "serious" interactions (hilarity). Q: URL?A: http:// svn.redbean.com/repos/ircbots/ Q: IM version? A: That'd be fun.
py.execnet: simple ad hoc networking Holger Kriekel Standard models for remote execution are cumbersome, require servers to be started ahead of time, and complicates programming. py.execnet requires no installation on the server side. You define your protocol on the client side and deploy it from there using a gateway via an SSH login or similar channel to a host with Python installed. Interactive demonstration showed /etc/passwd from a remote host. Great for propagating viruses ;-). In a controlled environment has been extremely useful for administrative tasks, and multiple scripts have been developed for tasks like remote sync. http://codespeak.net/py
Help with Jython Generator Expressions Chio??? I am trying to expect generator expressions in Jython. Sometimes it works, but sometimes it fails. If there's someone who knows Jython internals or AST would be able to help mne. Please!
Using the Web as Your Application Platform ??? We are taking REST to extremes. tasty, our tagging service, is a layer that provides tag search results as Python dictionaries. We have focused on small components, and have found this is a fruitful design approach. http://tasty.python-hosting.com.
Spec David Byng spec is a module we have been using to describe sets - think of it as a type. Allows checking for membership in sets, describing and checking attributes. Various set-theoretic primitives are implemented as functions. the proper() function can be used to verify that an instance agrees with a set of property specifications. http://www.mems-exchange.org/software/qp/. Q: Did you look at spyce(??)? A: It's kind of similar, but different enough to be necessary.
A Configuration Database Moshe Zadka The database has to be updated by all kinds of system components, and be available to many processes. A configuration wholuld be a list of actions, saying what the user did to get to the current state. We have a schema description language. This requires very little code. People are doing similar stuff with prevailer.
Another session tomorrow!!!!!
This is a stream-of-consciousness dump of the lightning talk sessions.
Big and Bold Slides with S5 David Goodger: With Python and docutils you can do large-slide presentations. Q: Can you print your slides? A: Yes, but the layout needs work. CSS anyone? Q: Can you include images? A: Yes, of all kinds? Q: Does it work with IE? A: Yes.
Testing Web Apps with Twill and ??? Titus ??? Twill lets you browse to web pages and fill out forms. Very engaging interactive demonstration. Nice humor: "...and of course the great thing about Python is there are so many different web frameworks you can test". I think I'm going to try twill. Q: Will that work with mod_python? A: It works client side, not server side.
Building MerchantCircle.com David Creemer: They are building a CheeryPy-based online marketing tool for small local merchants. Lots of Python in the application, so thanks. Found it bery good to develop in the production environment. Found FormEncode, Routes, PostgreSQL and Tsearch2 very useful. Avoid magic, be(come) a DBA, hire the best and never stop cleaning. Q: Did you use embedded Python stored procedures? A: We've played with it, but no. Q: Would you like something similar to routes in CherryPy? A: I'd love to see that.
Python Argetina Facundo Batista: We are a user group, and have a portal web site, recently migrated away from Plone to MoinMoin. We have developed a stable workshop in a national university, and given courses. We have T-shirts for sale. We have meetings, talk a lot and drink a lot! We are planning a national meeting to allow people from the different provinces to meet each other, and want to organize a job board, talks in various cities, and keep reaching out to Argentinian users. Q: How small are your smallest T-shirts? A: Medium, but htat's smaller that your medium! (laughter). We don't have XX (more laughter). Q: What other languages are the universities teaching? A: C++ and Java.
Chandler ??? Chandler handles calendar events now, shortll appointments and to-do lists, eventually email. It also lets you share information via a WebDAV repository. 0.6.1 is the latest release, and the speaker is using it live although there arae some rough edges. Built on wxPython and twisted. The GUI allows multiple calendars to be overlaid, which is neat. An event can occur in multiple calendars. Eventually they want Chandler to incorporate many extensions from the Python community. Q: License? A: It's an open source project, with an open license.
How to Replace Yourself with a Small Bot Ben Collins-Sussman After five years of daily IRC use the speaker started to use ircbot.py and generate bots of various kinds, starting with pinkybot and wolfbot. He moved to work with Google, and realised that a bot could take his place in the various communities, and sussbot was born. It does nothing except respond with useless platitudes such as "have you checked the FAQ?" little happened, but some users did undertake "serious" interactions (hilarity). Q: URL?A: http:// svn.redbean.com/repos/ircbots/ Q: IM version? A: That'd be fun.
py.execnet: simple ad hoc networking Holger Kriekel Standard models for remote execution are cumbersome, require servers to be started ahead of time, and complicates programming. py.execnet requires no installation on the server side. You define your protocol on the client side and deploy it from there using a gateway via an SSH login or similar channel to a host with Python installed. Interactive demonstration showed /etc/passwd from a remote host. Great for propagating viruses ;-). In a controlled environment has been extremely useful for administrative tasks, and multiple scripts have been developed for tasks like remote sync. http://codespeak.net/py
Help with Jython Generator Expressions Chio??? I am trying to expect generator expressions in Jython. Sometimes it works, but sometimes it fails. If there's someone who knows Jython internals or AST would be able to help mne. Please!
Using the Web as Your Application Platform ??? We are taking REST to extremes. tasty, our tagging service, is a layer that provides tag search results as Python dictionaries. We have focused on small components, and have found this is a fruitful design approach. http://tasty.python-hosting.com.
Spec David Byng spec is a module we have been using to describe sets - think of it as a type. Allows checking for membership in sets, describing and checking attributes. Various set-theoretic primitives are implemented as functions. the proper() function can be used to verify that an instance agrees with a set of property specifications. http://www.mems-exchange.org/software/qp/. Q: Did you look at spyce(??)? A: It's kind of similar, but different enough to be necessary.
A Configuration Database Moshe Zadka The database has to be updated by all kinds of system components, and be available to many processes. A configuration wholuld be a list of actions, saying what the user did to get to the current state. We have a schema description language. This requires very little code. People are doing similar stuff with prevailer.
Another session tomorrow!!!!!
Implementing IronPython
Again another full talk, this time from Jim Hugunin, author of both Jython and IronPython. Jim summarised the state of the CLR, pointing out that it's now a standard. Jim enjoys seeing the IronPython download link on a web page with the Visual Studio icon on it.
The last year has been spent on ensuring that IronPython can pass as many as possible of the standard Python regression tests. Certain tests have been modified, because they test implementation optimizations from CPython. There are some flat differences (like IronPython doesn't use refcounting). Some are arguable, such as the use of Unicode attribute names, which people do expect to be able to do in the CLR world.
The structure of the IronPython compiler is now (since CPython's AST branch was implemented) closely aligned with the CPython compiler, but of course it produces IL operations for the CLR rather than the Python bytecodes of CPython. Python bytecodes are simpler than IL bytecodes, byt there is a significant speed gan from the CLR's use of JIT compilation into machine code. Not every operation can run 100 times as fast on IronPython, but some operations become a single machine-code instruction. Jim wishes he could get that speedup for all of IronPython.
He demonstrated some of the tools you can use to examine the generated code, unfortunately without identifying them. It seems relatively straightforward, which will surely be helpful to other implementers and more advanced users. Rather than keep a complicated line number table, no-ops are inserted into the code. Jim contrasted his code generator with that of the C implementation, and I was impressed with the simplicity of the code generation code.
Jim wanted to be able to claim that IronPython used a "simpler" object layout than CPython - you might expect that because no reference count is required in a true grabage collection environment. Unfortunately they have had to give each object a 32-bit "synch block" that holds arbitrary data to assist various implementation features, so integers are 12 bytes (presumably for 2-bit machines) in both systems. List representations are remarkably similar, but IronPython does seem to save a little space there.
The type pointer is different for IronPython, in an attempt to maintain performance. The first sensible way to proceed is to wrap every external object with a subclass of some sort of Python PyObject (this approach was taken by Jython). This requires a lot of wrappping and unwrapping during method calls. The other way, used in IronPython, is to used a "pure object model", allowing the underlying objects' methods to be called directly and the returned values to be returned directly.
Subtyping a builtin class permits changing the type of an instance by changing the instance's __class__ slot, but the type as perceived by the CLR cannot change.
Fields and proerties of CLR objects are closely parallel to descriptors. Overloaded methods analyse their arguments to disambiguate their signatures. Constructors are converted to __new__(). And so on (Jim was goung fast here). The Python object model turns out to be surprisingly convenient for implementation on CLR, which Jim felt was a tribute to Guido's design skills.
Jim demonstrated an interactive session in which he created a window in the presentation framework. He then defined a CallMe class and bound an instance of the class to a button's click event. A panel subclass was defined, and an instance rendered. Jim's bravery in performing live reali-time coding is much to be admired, and says a lto about his confidence and knowledge. Overall a very convincing demonstration that IronPython brings true Pythonicity to the CLR environment.
Jim closed with an interesting discussion about compatibility. Should IronPython expose CLR native object methods as methods of Python objects? Both sides, when Jim talked to them, felt that their conflicting opinions were the only obvious answer to the question! His attempt to please the CLR folks has led to the development of a clr module. import clr changes method call semantics change to expose the CLR methods. Jim had a lot of pushback from the CLR world not to use the from __future__ notation for this feature. The fortunate difference in naming conventions between the two environments means that there are very few conflicts.
Exception handling is another are where the two environments conflict. There are two class hierarchies to be resolved. If you try to catch a .NET exception then the corresponding Pythnon exception will be converted.
Wow! Now I have to get lunch ...
The last year has been spent on ensuring that IronPython can pass as many as possible of the standard Python regression tests. Certain tests have been modified, because they test implementation optimizations from CPython. There are some flat differences (like IronPython doesn't use refcounting). Some are arguable, such as the use of Unicode attribute names, which people do expect to be able to do in the CLR world.
The structure of the IronPython compiler is now (since CPython's AST branch was implemented) closely aligned with the CPython compiler, but of course it produces IL operations for the CLR rather than the Python bytecodes of CPython. Python bytecodes are simpler than IL bytecodes, byt there is a significant speed gan from the CLR's use of JIT compilation into machine code. Not every operation can run 100 times as fast on IronPython, but some operations become a single machine-code instruction. Jim wishes he could get that speedup for all of IronPython.
He demonstrated some of the tools you can use to examine the generated code, unfortunately without identifying them. It seems relatively straightforward, which will surely be helpful to other implementers and more advanced users. Rather than keep a complicated line number table, no-ops are inserted into the code. Jim contrasted his code generator with that of the C implementation, and I was impressed with the simplicity of the code generation code.
Jim wanted to be able to claim that IronPython used a "simpler" object layout than CPython - you might expect that because no reference count is required in a true grabage collection environment. Unfortunately they have had to give each object a 32-bit "synch block" that holds arbitrary data to assist various implementation features, so integers are 12 bytes (presumably for 2-bit machines) in both systems. List representations are remarkably similar, but IronPython does seem to save a little space there.
The type pointer is different for IronPython, in an attempt to maintain performance. The first sensible way to proceed is to wrap every external object with a subclass of some sort of Python PyObject (this approach was taken by Jython). This requires a lot of wrappping and unwrapping during method calls. The other way, used in IronPython, is to used a "pure object model", allowing the underlying objects' methods to be called directly and the returned values to be returned directly.
Subtyping a builtin class permits changing the type of an instance by changing the instance's __class__ slot, but the type as perceived by the CLR cannot change.
Fields and proerties of CLR objects are closely parallel to descriptors. Overloaded methods analyse their arguments to disambiguate their signatures. Constructors are converted to __new__(). And so on (Jim was goung fast here). The Python object model turns out to be surprisingly convenient for implementation on CLR, which Jim felt was a tribute to Guido's design skills.
Jim demonstrated an interactive session in which he created a window in the presentation framework. He then defined a CallMe class and bound an instance of the class to a button's click event. A panel subclass was defined, and an instance rendered. Jim's bravery in performing live reali-time coding is much to be admired, and says a lto about his confidence and knowledge. Overall a very convincing demonstration that IronPython brings true Pythonicity to the CLR environment.
Jim closed with an interesting discussion about compatibility. Should IronPython expose CLR native object methods as methods of Python objects? Both sides, when Jim talked to them, felt that their conflicting opinions were the only obvious answer to the question! His attempt to please the CLR folks has led to the development of a clr module. import clr changes method call semantics change to expose the CLR methods. Jim had a lot of pushback from the CLR world not to use the from __future__ notation for this feature. The fortunate difference in naming conventions between the two environments means that there are very few conflicts.
Exception handling is another are where the two environments conflict. There are two class hierarchies to be resolved. If you try to catch a .NET exception then the corresponding Pythnon exception will be converted.
Wow! Now I have to get lunch ...
Implementation of the Python ByteCode Compiler
My primary interest in this talk was hearing Jeremy Hylton speak. Jeremy is a formidable intellect, and has made many fine contributions to the Python core. It turns out he is alsi an excellent (if somewhat rapid-fire) speaker.
He introduced the abstract syntax tree notation, showing how it contrasted with the previously-used concrete tree, then went on to look at compilation strategy.
The first pass builds a symbol table, and collects certain information (for example, if a function contains a yield then it's a generator). A second pass determines the scope of the names, with some unexpected differences between implcit and explicit global variables (whose treatment differs, for example, in the class namespace).
Free variables must be treated differently from locals, because their lifetime can extend beyond the activation record of a function call. There are many fine points to be considered in the handling of names, and some of Jeremy's examples were chosen to stress the syntax. Sadly they zipped by so fast you'll have to read the slides for the details, but you can consider his first example
Jeremy's treatment of code generation was concise (he clearly wanted to cover a lot of ground, so he wasn't hanging about), but I did at least understand that the output of an AST tree transformation was a set of basic blocks, pointers to which would be jump targets. Then the code generator iterates over the contents of each block, emitting the bytecodes for each AST type encountered, essentially as a visiting tree walk.
An assembler then linearizes the resulting code blocks, computing the required stack space and the required entries in a line number table (whose construction is horrendously complicated). Jump offsets can be deduced at this point, and then PyCode_New() is called to create the object. Finally the peephole optimizer cherry-picks the resulting code, performing constant folding and simplifying certain jump sequences down to a single jump.
The AST branch has simplified several aspects of the compiler implementation, but it also has potential value in other areas. If the AST code could be exposed to Python applications, for example, tools like pychecker could use the tree rather than the source. The application interface has not yet been determined, and since the core currently contains code to handle the concrete syntax tree there will be library ramifications, especially in the compiler package.
Q: If an application uses the AST, do the nodes make the original program text available?
A: For names, yes, but for example floating point objects have already been converted. Statements and expressions refer to their line numbers.
Q: Can I reproduce the source of a program from the AST?
A: That's something that we need to work on to support applications like a refactoring tool, for which the current implementation would be insufficient.
Q: LOAD_NAME only appears to be used in exec and class. Is it used anywhere else?
A: I don't know. [Compiles code and disassembles it]. Ah, yes!
Q: Has anyone considered [inaudible mumble].
A: I don't think so, but that would be a more complicated but more traditional way to do it.
Q: Can you talk about performance a little?
A: Yes. Thanks to Neal Norwitz, Neal Schemenauer, Brett Cannon and Tim Peters the new compiler is marginally faster than the old. Our primary goal was to write comprehensible code, however, so there may well be further gains. For example the peephole optimizer should really be run before the assembler.
Not really sure whether this is any use to someone who wasn't there!
He introduced the abstract syntax tree notation, showing how it contrasted with the previously-used concrete tree, then went on to look at compilation strategy.
The first pass builds a symbol table, and collects certain information (for example, if a function contains a yield then it's a generator). A second pass determines the scope of the names, with some unexpected differences between implcit and explicit global variables (whose treatment differs, for example, in the class namespace).
Free variables must be treated differently from locals, because their lifetime can extend beyond the activation record of a function call. There are many fine points to be considered in the handling of names, and some of Jeremy's examples were chosen to stress the syntax. Sadly they zipped by so fast you'll have to read the slides for the details, but you can consider his first example
class Spam:as typical.
id = id(1)
Jeremy's treatment of code generation was concise (he clearly wanted to cover a lot of ground, so he wasn't hanging about), but I did at least understand that the output of an AST tree transformation was a set of basic blocks, pointers to which would be jump targets. Then the code generator iterates over the contents of each block, emitting the bytecodes for each AST type encountered, essentially as a visiting tree walk.
An assembler then linearizes the resulting code blocks, computing the required stack space and the required entries in a line number table (whose construction is horrendously complicated). Jump offsets can be deduced at this point, and then PyCode_New() is called to create the object. Finally the peephole optimizer cherry-picks the resulting code, performing constant folding and simplifying certain jump sequences down to a single jump.
The AST branch has simplified several aspects of the compiler implementation, but it also has potential value in other areas. If the AST code could be exposed to Python applications, for example, tools like pychecker could use the tree rather than the source. The application interface has not yet been determined, and since the core currently contains code to handle the concrete syntax tree there will be library ramifications, especially in the compiler package.
Q: If an application uses the AST, do the nodes make the original program text available?
A: For names, yes, but for example floating point objects have already been converted. Statements and expressions refer to their line numbers.
Q: Can I reproduce the source of a program from the AST?
A: That's something that we need to work on to support applications like a refactoring tool, for which the current implementation would be insufficient.
Q: LOAD_NAME only appears to be used in exec and class. Is it used anywhere else?
A: I don't know. [Compiles code and disassembles it]. Ah, yes!
Q: Has anyone considered [inaudible mumble].
A: I don't think so, but that would be a more complicated but more traditional way to do it.
Q: Can you talk about performance a little?
A: Yes. Thanks to Neal Norwitz, Neal Schemenauer, Brett Cannon and Tim Peters the new compiler is marginally faster than the old. Our primary goal was to write comprehensible code, however, so there may well be further gains. For example the peephole optimizer should really be run before the assembler.
Not really sure whether this is any use to someone who wasn't there!
State of the Python Universe
Since many people will be commenting on this presentation, I'll see if I can't just try to summarise for those who want the headlines without having to read the whole story.
The Move to Subversion - the main reason for the move was the slowness of SourceForge. [Apparently Guido got a message yesterday to say that SF do now have Subversion available]. I wonder how many projects that didn't have the resources to move to their own SVN repository are still struggling with delays before their commits appear in the anonymously-available versions. Maybe SVN on SF has helped.
Buildbot - with the introduction of this system we expect to see big improvements in the detection of single-platform errors and checkins performed without passing tests.
The Cheese Shop - has been a great step forward in Python non-core code availability. More people should be using it to release their code.
www.python.org - would you like to be a python.orb webmaster? We are looking for helpers, and the new design was specifically intended to make content management easier. Email to pydotorg@python.org if you'd like to help.
Python 2.5 release schedule - expect to see 2.5 around September 30. There will be a 2.4.3 bugfix release before the first alpha of the new release, and a final 2.4.4 (barring real security problems) after the 2.5 dust has cleared.
The import mechanism will change, introducing a mechanism to specify import from a position relative to the current package.
The new conditional expressions have a slightly quirky syntax, and it sounds as though Guido has made this choice to discourage its over-use. The essence of his remarks was that it's not Pythonic (though he didn't use that word) to write too many lengthy expressions spanning multiple lines.
It will become possible to write exception handling constructs with try ... except ... finally rather than having two nested constructs. One of the advantages of Guido's time spent in Java-jail is that he realised after reading the Java documentation that this does not introduce any ambiguity.
Generator semantics will change to allow the yield keyword to return a value rather than introducing a statement. This value is returned by calling the generator's .send() method. It's actually possible to cause the yield to raise an exception by calling the generator's .throw() method. Generators also have a .close() method that raises a GeneratorExit exception. This new method will be closed when the generator is garbage-collected.
The with statement is intended to ease the coding of critical sections which require both initialization and clean-up. with is followed by an expression whose value's __context__() method returns a value that has .__enter__() and .__exit__() methods. The expression is known as the context manager and in the majority of cases __context__() can return self, allowing all three methods to be defined in a single class. Some context managers do need to be independent, however, which is why the semantics were defined this way. Certain standard objects will be redefined as context managers to allow their use in with statements, the major ones being mutexes and lock, decimal objects and files.
Exceptions are finally going to become new-style classes, with the inheritance tree rooted at the new BaseException. The end of string exceptions is now in sight. This may eventually lead to a bare except clause becoming illegal (sensible, because this forces you to use an interface in the sys module to access the exception). User-defined exceptions will continue to use Exception as their base class, but KeyboardInterrupt and SystemExit will no longer be subclasses of Exception, allowing cleaner handling of the mainstream exceptions.
Types will be able to define an __index__() method, guaranteed to return an integer or long value. This has been introduced principally to meet the requirements for slicing of complex structures, where the __int__() method isn't appropriate.
The compiler has been revised to create an abstract syntax tree rather than a concrete one. This was a development that almost didn't make it, after much work. Only after Guido said "let's forget it" did the developers realise that they didn't want to lose results of the hard work and finally integrate it into the trunk. This has led to improvements in the maintainability of the compiler, allowing developers to make experimental syntax changes more easily.
Other highlights - Pretty much invisible to users, but invaluable for C developers on 64-bit machines, is the new ssize_t type. This will allow strings (and even lists!) to be longer than 2G elements. The -m command-line option will now allow you to run package submodules as well as top-level modules. This has already proved its value in running regression tests.
Python eggs support is at least on the horizon, making simpler installation a practical possibility for the average Python user.
A new dictionary object, collections.defaultdict, will allow much more flexibility and efficiency for certain dictionary applications.
lambda will survive beyond Python 2.x because "it's perfect" (loud applause). Because it's perfect, no changes will be made.
Q: Will Python access to the AST allow, for example, the implementation of LISP-style macros?
A: I suppose it could be abused for that. (AMK, after galloping to the microphone: The AST BoF session will be discussing this topic at 3:15)
Q: Was "exception filtering" considered?
A: No
Q: Can you add a __missing__ method to a dict?
A: No, builtin types can't be modified in that way.
Q: What happens when two generators' finally clauses call each other?
A: I have no idea. There's a fair chance that it will work, but pathological cases can doubtless be constructed. Don't try and play games with this: you won't even understand what it means yourself!
Q: Can a defaultdict's factory function access the key value?
A: No. To pass the key to the factory function would slow things down. You'll have to override __missing__ in a subclass to do this, as the standard implementation is focused on the common use cases.
Q: How do you like working at Google?
A: Ask me during the break. I love it!
The Move to Subversion - the main reason for the move was the slowness of SourceForge. [Apparently Guido got a message yesterday to say that SF do now have Subversion available]. I wonder how many projects that didn't have the resources to move to their own SVN repository are still struggling with delays before their commits appear in the anonymously-available versions. Maybe SVN on SF has helped.
Buildbot - with the introduction of this system we expect to see big improvements in the detection of single-platform errors and checkins performed without passing tests.
The Cheese Shop - has been a great step forward in Python non-core code availability. More people should be using it to release their code.
www.python.org - would you like to be a python.orb webmaster? We are looking for helpers, and the new design was specifically intended to make content management easier. Email to pydotorg@python.org if you'd like to help.
Python 2.5 release schedule - expect to see 2.5 around September 30. There will be a 2.4.3 bugfix release before the first alpha of the new release, and a final 2.4.4 (barring real security problems) after the 2.5 dust has cleared.
The import mechanism will change, introducing a mechanism to specify import from a position relative to the current package.
The new conditional expressions have a slightly quirky syntax, and it sounds as though Guido has made this choice to discourage its over-use. The essence of his remarks was that it's not Pythonic (though he didn't use that word) to write too many lengthy expressions spanning multiple lines.
It will become possible to write exception handling constructs with try ... except ... finally rather than having two nested constructs. One of the advantages of Guido's time spent in Java-jail is that he realised after reading the Java documentation that this does not introduce any ambiguity.
Generator semantics will change to allow the yield keyword to return a value rather than introducing a statement. This value is returned by calling the generator's .send() method. It's actually possible to cause the yield to raise an exception by calling the generator's .throw() method. Generators also have a .close() method that raises a GeneratorExit exception. This new method will be closed when the generator is garbage-collected.
The with statement is intended to ease the coding of critical sections which require both initialization and clean-up. with is followed by an expression whose value's __context__() method returns a value that has .__enter__() and .__exit__() methods. The expression is known as the context manager and in the majority of cases __context__() can return self, allowing all three methods to be defined in a single class. Some context managers do need to be independent, however, which is why the semantics were defined this way. Certain standard objects will be redefined as context managers to allow their use in with statements, the major ones being mutexes and lock, decimal objects and files.
Exceptions are finally going to become new-style classes, with the inheritance tree rooted at the new BaseException. The end of string exceptions is now in sight. This may eventually lead to a bare except clause becoming illegal (sensible, because this forces you to use an interface in the sys module to access the exception). User-defined exceptions will continue to use Exception as their base class, but KeyboardInterrupt and SystemExit will no longer be subclasses of Exception, allowing cleaner handling of the mainstream exceptions.
Types will be able to define an __index__() method, guaranteed to return an integer or long value. This has been introduced principally to meet the requirements for slicing of complex structures, where the __int__() method isn't appropriate.
The compiler has been revised to create an abstract syntax tree rather than a concrete one. This was a development that almost didn't make it, after much work. Only after Guido said "let's forget it" did the developers realise that they didn't want to lose results of the hard work and finally integrate it into the trunk. This has led to improvements in the maintainability of the compiler, allowing developers to make experimental syntax changes more easily.
Other highlights - Pretty much invisible to users, but invaluable for C developers on 64-bit machines, is the new ssize_t type. This will allow strings (and even lists!) to be longer than 2G elements. The -m command-line option will now allow you to run package submodules as well as top-level modules. This has already proved its value in running regression tests.
Python eggs support is at least on the horizon, making simpler installation a practical possibility for the average Python user.
A new dictionary object, collections.defaultdict, will allow much more flexibility and efficiency for certain dictionary applications.
lambda will survive beyond Python 2.x because "it's perfect" (loud applause). Because it's perfect, no changes will be made.
Q: Will Python access to the AST allow, for example, the implementation of LISP-style macros?
A: I suppose it could be abused for that. (AMK, after galloping to the microphone: The AST BoF session will be discussing this topic at 3:15)
Q: Was "exception filtering" considered?
A: No
Q: Can you add a __missing__ method to a dict?
A: No, builtin types can't be modified in that way.
Q: What happens when two generators' finally clauses call each other?
A: I have no idea. There's a fair chance that it will work, but pathological cases can doubtless be constructed. Don't try and play games with this: you won't even understand what it means yourself!
Q: Can a defaultdict's factory function access the key value?
A: No. To pass the key to the factory function would slow things down. You'll have to override __missing__ in a subclass to do this, as the standard implementation is focused on the common use cases.
Q: How do you like working at Google?
A: Ask me during the break. I love it!
Industrial Light and Miserliness?
Does LucasFilm Division Have Hidden Financial Troubles?
I guess you don't get rich by giving money away. That, anyway, is my conclusion after a little research into the personal and corporate finances of George Lucas, scion of the film industry. Forbes magazine regularly assesses his personal wealth at $1 billion plus, but one of his companies appears to be suffering unacknowledged financial hardship.
Last May George's company LucasFilm moved to a new $350 million headquarters building at the Presidio, allowing them to "be more efficient and leverage assets, technology people and innovations that come from every corner of the company", as he apparently said at the time.
According to industry analysts the difficulty until then had been that the separate divisions of the Lucas empire were located at disparate locations in Marin County. According to LucasFilm's President, Micheline Chau, "Marin has been really tough on us, we ran our businesses as silos". Since July the effects and animation division, Industrial Light and Magic, has joined the other LucasFilm divisions at the new headquarters.
The advantage of this move is that LucasFilm can now use the same assets they created for a film to produce the spinoff game and other ancillary products. This can be big business - it's reckoned that even running as separate silos the 2003 turnover at LucasFilm ran to $1.2 billion, which certainly lends a new meaning to the word "tough". Estimates are all we have, of course, because private companies like LucasFilm don't have to publish accounts or have their books scrutinized.
It's a good job that the company appears to be doing well overall. Yesterday's Python Software Foundation members' meeting reminded me that two years ago we were asked to vote to admit Industrial Light and Magic as a sponsor member of the Foundation (sponsor members pay annual dues, currently $2,000, to help fund the Foundation's growth). Their membership was approved by 42 votes to 1, mine being the sole dissenting voice. This was not a serious attempt to block ILM's membership but primarily because I felt that a company like ILM could have been doing much more to support the Foundation.
I have heard (from staffers I have met at conferences) that the company employs in excess of seven hundred Python programmers. Here's what Tommy Burnette, ILM's Senior Technical Director, is on record as saying "Python plays a key role in our production pipeline. Without it a project the size of Star Wars: Episode II would have been very difficult to pull off. From our crowd rendering to batch processing to compositing, Python binds all things together." Philip Peterson, a Principal Engineer for R&D adds "Python is everywhere at ILM. It's used to extend the capabilities of our applications, as well as providing the glue between them. Every CG image we create has involved Python somewhere in the process".
This is clearly an organization that makes extensive use of Python, and one imagines it would profit by it (as the PSF hopes all users profit by their use of Python). Sadly this appears not to be the case. When you look at the PSF members roster, you may wonder why you see ILM listed at the bottom of the page as an emeritus member rather than a sponsor member.
This is where my incisive intelligence tells me the company must have been through bad financial troubles. Shortly after the membership voted to accept ILM as a member a PSF director received a 'phone call from our contact there explaining that they could not find $2,000 in their budget to pay their membership dues. Since by that time they had been elected to membership our only choice under the existing constitution was to convert their membership to the non-paying non-voting emeritus status normally reserved for retired and inactive individual members.
I'm not one to bear a grudge, and I hope that ILM's financial position has improved since then to the point where they can not only afford the fees to become a sponsor member, but also provide Platinum level sponsorship for PyCon like Google, ITA Software and EWT have done this year. I have a suspicion that if they looked really hard they might find that they had good reason to help the Foundation that Guido van Rossum created to administer the intellectual property that he and the other Python developers have created. We need more supporters with the experience and clout of Industrial Light and Magic, and I think we should welcome them into the fold.
These ponderings also raise a larger question: since FLOSS (Free/Libre Open Source Software) is produced and distributed essentially at no cost, what (if any) obligation do its users have to support the development process? Clearly there's nothing that can be done directly to force any return on development efforts, and most open source contributors I know would rather be wrting code (or even documentation!) than spending time policing payback. Many open source users contribute effectively to open source projects by having staff work as developers, so their work is incorporated into the intellectual property of the project. Others prefer to donate to supporting foundations and similar bodies.
If you think about the larger context of open source, though, there's a sea change afoot. In today's world the proprietary model dominates most of the software market, but open source is gaining ground. Large companies have responded to the change in various different ways. There's almost a paradox about open source development in a capitalist economy: if everyone uses open source software without taking steps to ensure that a project keeps going, it will falter and die. You might also argue that this is also quite Darwinian: if a project is sufficiently important to a sufficiently wealthy section of the user universe, it will prosper and grow.
I don't think anyone has a clear idea of where this is all going. It will be interesting to see what eventually transpires.
[Steve Holden is a member of the Python Software Foundation Board]
I guess you don't get rich by giving money away. That, anyway, is my conclusion after a little research into the personal and corporate finances of George Lucas, scion of the film industry. Forbes magazine regularly assesses his personal wealth at $1 billion plus, but one of his companies appears to be suffering unacknowledged financial hardship.
Last May George's company LucasFilm moved to a new $350 million headquarters building at the Presidio, allowing them to "be more efficient and leverage assets, technology people and innovations that come from every corner of the company", as he apparently said at the time.
According to industry analysts the difficulty until then had been that the separate divisions of the Lucas empire were located at disparate locations in Marin County. According to LucasFilm's President, Micheline Chau, "Marin has been really tough on us, we ran our businesses as silos". Since July the effects and animation division, Industrial Light and Magic, has joined the other LucasFilm divisions at the new headquarters.
The advantage of this move is that LucasFilm can now use the same assets they created for a film to produce the spinoff game and other ancillary products. This can be big business - it's reckoned that even running as separate silos the 2003 turnover at LucasFilm ran to $1.2 billion, which certainly lends a new meaning to the word "tough". Estimates are all we have, of course, because private companies like LucasFilm don't have to publish accounts or have their books scrutinized.
It's a good job that the company appears to be doing well overall. Yesterday's Python Software Foundation members' meeting reminded me that two years ago we were asked to vote to admit Industrial Light and Magic as a sponsor member of the Foundation (sponsor members pay annual dues, currently $2,000, to help fund the Foundation's growth). Their membership was approved by 42 votes to 1, mine being the sole dissenting voice. This was not a serious attempt to block ILM's membership but primarily because I felt that a company like ILM could have been doing much more to support the Foundation.
I have heard (from staffers I have met at conferences) that the company employs in excess of seven hundred Python programmers. Here's what Tommy Burnette, ILM's Senior Technical Director, is on record as saying "Python plays a key role in our production pipeline. Without it a project the size of Star Wars: Episode II would have been very difficult to pull off. From our crowd rendering to batch processing to compositing, Python binds all things together." Philip Peterson, a Principal Engineer for R&D adds "Python is everywhere at ILM. It's used to extend the capabilities of our applications, as well as providing the glue between them. Every CG image we create has involved Python somewhere in the process".
This is clearly an organization that makes extensive use of Python, and one imagines it would profit by it (as the PSF hopes all users profit by their use of Python). Sadly this appears not to be the case. When you look at the PSF members roster, you may wonder why you see ILM listed at the bottom of the page as an emeritus member rather than a sponsor member.
This is where my incisive intelligence tells me the company must have been through bad financial troubles. Shortly after the membership voted to accept ILM as a member a PSF director received a 'phone call from our contact there explaining that they could not find $2,000 in their budget to pay their membership dues. Since by that time they had been elected to membership our only choice under the existing constitution was to convert their membership to the non-paying non-voting emeritus status normally reserved for retired and inactive individual members.
I'm not one to bear a grudge, and I hope that ILM's financial position has improved since then to the point where they can not only afford the fees to become a sponsor member, but also provide Platinum level sponsorship for PyCon like Google, ITA Software and EWT have done this year. I have a suspicion that if they looked really hard they might find that they had good reason to help the Foundation that Guido van Rossum created to administer the intellectual property that he and the other Python developers have created. We need more supporters with the experience and clout of Industrial Light and Magic, and I think we should welcome them into the fold.
These ponderings also raise a larger question: since FLOSS (Free/Libre Open Source Software) is produced and distributed essentially at no cost, what (if any) obligation do its users have to support the development process? Clearly there's nothing that can be done directly to force any return on development efforts, and most open source contributors I know would rather be wrting code (or even documentation!) than spending time policing payback. Many open source users contribute effectively to open source projects by having staff work as developers, so their work is incorporated into the intellectual property of the project. Others prefer to donate to supporting foundations and similar bodies.
If you think about the larger context of open source, though, there's a sea change afoot. In today's world the proprietary model dominates most of the software market, but open source is gaining ground. Large companies have responded to the change in various different ways. There's almost a paradox about open source development in a capitalist economy: if everyone uses open source software without taking steps to ensure that a project keeps going, it will falter and die. You might also argue that this is also quite Darwinian: if a project is sufficiently important to a sufficiently wealthy section of the user universe, it will prosper and grow.
I don't think anyone has a clear idea of where this is all going. It will be interesting to see what eventually transpires.
[Steve Holden is a member of the Python Software Foundation Board]
February 24, 2006
Understanding Unicode
David Goodger set himself the possibly ambitious goal of dispersing unicodaphobia, which he defined as fear of Unicode and all the words surrounding it. Unicode is intended to be a cross-platform standard applicable to any program and any language (possibly even including Dwarvish and Klingon in the future). It is currently defined by ISO 10646. What follows is my attempt to record my developing understanding during the presentation.
The basic element in Unicode is the character: a character set takes a subset of the characters and maps each one to a code point, a numeric value associated (in that particular character set) with that character. There are a number of 8-bit character sets that all associate the ASCII characters with the code points from 0-127, but mapping completely different characters onto the upper half of the code points.
Unlike Western alphabets, many other languages have large characters sets that therefore cannot be mapped onto only 256 code points.
An encoding is a specification of the code points mapped to each of the characters in a character set. An encoding will be performed by a codec. Encoding a Unicode text maps each character in a Unicode text into one or more bytes in a representation. Decoding is the converse process. in a Python program the internal representation might use 16 or 32 bits per character, depending on the build configuration.
Whatever the internal representation of the Unicode text, text that is read in must be decioded from its external representation to the internal representation, and when a Unicode string is written out it must be encoded into a suitable encoding (but not necessarily the same one that was used to read it in). ASCII is an encoding, as is Latin-1 throughLatin-N and Windows-1252; all thses encodings are ASCII-compatible because each code point is a single byte containing the ASCII set on code points 0 through 127. UCS-2, UCS-4 and UTF-16 are all encodings that are capable of representing all Unicode characters. Because they use multi-byte code points they aren't ASCII-compatible.
The new default encoding is UTF-8, which is ASCII-compatible because the characters of the ASCII set still map onto code points represented as single bytes with values of 1 through 127. It is capable of representing all Unicode characters, because it uses multi-byte representations (from 1 to 6 bytes in length) for the non-ASCII characters, and none of the bytes in those representations are in the range 1 through 127. The first byte of a non-ASCII character is always in the range 0xc0 to 0xFD, and the second byte will be in the range 0x80 to 0xBF.
For this student the main failing of the presentation was the complete absence of any graphical materials. Some of us understand pictures better than just words. I was left with the feeling that my understanding of Unicode had increased, but I couldn't necessarily say I can proceed confidently from here on in. I wish the presentation hadn't overrun its time, as the noise of delegates departing early distracted from the later slides.
The basic element in Unicode is the character: a character set takes a subset of the characters and maps each one to a code point, a numeric value associated (in that particular character set) with that character. There are a number of 8-bit character sets that all associate the ASCII characters with the code points from 0-127, but mapping completely different characters onto the upper half of the code points.
Unlike Western alphabets, many other languages have large characters sets that therefore cannot be mapped onto only 256 code points.
An encoding is a specification of the code points mapped to each of the characters in a character set. An encoding will be performed by a codec. Encoding a Unicode text maps each character in a Unicode text into one or more bytes in a representation. Decoding is the converse process. in a Python program the internal representation might use 16 or 32 bits per character, depending on the build configuration.
Whatever the internal representation of the Unicode text, text that is read in must be decioded from its external representation to the internal representation, and when a Unicode string is written out it must be encoded into a suitable encoding (but not necessarily the same one that was used to read it in). ASCII is an encoding, as is Latin-1 throughLatin-N and Windows-1252; all thses encodings are ASCII-compatible because each code point is a single byte containing the ASCII set on code points 0 through 127. UCS-2, UCS-4 and UTF-16 are all encodings that are capable of representing all Unicode characters. Because they use multi-byte code points they aren't ASCII-compatible.
The new default encoding is UTF-8, which is ASCII-compatible because the characters of the ASCII set still map onto code points represented as single bytes with values of 1 through 127. It is capable of representing all Unicode characters, because it uses multi-byte representations (from 1 to 6 bytes in length) for the non-ASCII characters, and none of the bytes in those representations are in the range 1 through 127. The first byte of a non-ASCII character is always in the range 0xc0 to 0xFD, and the second byte will be in the range 0x80 to 0xBF.
For this student the main failing of the presentation was the complete absence of any graphical materials. Some of us understand pictures better than just words. I was left with the feeling that my understanding of Unicode had increased, but I couldn't necessarily say I can proceed confidently from here on in. I wish the presentation hadn't overrun its time, as the noise of delegates departing early distracted from the later slides.
Plone IS Easy to Install
It turns out they weren't lying. I downloaded the Plone installer, and now have an instance running on port 1080. Way to go, guys!
Using Django to Supercharge Web Development
A nicely judged talk with pleasantly humorous diversions.
Django is intended to enable the creation of in-depth web sites such as washingtonpost.com, and Adrian Holovaty gave an example (lawrence.com) which certainly proved the point. He explained that another project had developed a web site to process reader feedback on the presidential debates ... in four hours! Since the roots of Django are in the newspaper world, they remind themselves that "the ink is never dry" on the web, and sites are subject to continual development.
After driving themselves stupid with PHP they discovered Python, fell in love, and set themselves the goals of:
1. Making web development stupidly fast
2. Automating the repetitive stuff
3. Practising loose coupling
4. Following best practices
5. Being efficient
An early (three-day!) project involved taking the local little league and building a web site featuring much of the same information available about major league teams. The kusports.com sites allows subscribers to sign up for baseball stats as short messages tot heir cellphones. Many other examples provided convincing evidence that Django is indeed versatile.
Last year (at PyCon 2005, when Djanfo was about 18 months old) Adrian gave a presentation on what was then "the CMS", and received a lot of feedback that suggested an open source distribution would be popular. Django is built around:
A URL dispatcher
A database wrapper
A template system
An admin framework, and
"Other pain relievers".
Django URLs are designed to be pleasant and readable. The scheme allows the programmer to write Python code that directly maps the URLs to program callables. This decouples the URL from the business logic. The callables, known as "views" in Django terminology, take a request object as their first argument (possibly with values extracted from the URL as additional arguments) and returns and HttpResponse object.
The usual development model is to design the database. Django helps you to create and populate the tables (though it doesn't stop you from using SQL if that turns out to be easier). The design process also results in the automatic generation of admin functionality that can be used imeediately by editorial staff to manage content. The Django team is fortunate to have a designer to build great interface pages.
The initial design is now production-ready, and the team can write the views and templates to produce the required pages. The templating language was deliberately kept simple (avoiding XML and programming languages) to maximize its utility to the designers.
The framework is fully ready for internationalization (the first add-on contributed to the project after it was open-sourced), and provides three different levels of caching: per-site, per-view and low-level (this latter being performed in raw Python).
Repeatedly-required view functionalities have been abstracted into "generic views" that are automatically available to developers with no further formality. Sites using Django include chicagocrime.org (an award-winning site that integrates with Google maps), washingtonpost.com (many feeds are provided by screen-scraping Python scripts), the Ellington system (an online publishing system for news and entertainment sites) and traincheck, a site you can SMS to find out when and where to catch your metro.
The 1.0 release is approaching, with much work upcoming in a post-Python sprint, and a book is expected later in the year from APress.
A nice final touch was the announcement of the availability of t-shirts featuring the slogan
It was great to see enthusiastic and capable developers using Python for pragmatic purposes. Django does not attempt to be a fully-integrated content-management framework, but a loosely-coupled collection of components that work extremely effectively together to provide extremely efficient mechanisms for buildding database-driven web sites. Go to djangoproject.com and take a look!
Django is intended to enable the creation of in-depth web sites such as washingtonpost.com, and Adrian Holovaty gave an example (lawrence.com) which certainly proved the point. He explained that another project had developed a web site to process reader feedback on the presidential debates ... in four hours! Since the roots of Django are in the newspaper world, they remind themselves that "the ink is never dry" on the web, and sites are subject to continual development.
After driving themselves stupid with PHP they discovered Python, fell in love, and set themselves the goals of:
1. Making web development stupidly fast
2. Automating the repetitive stuff
3. Practising loose coupling
4. Following best practices
5. Being efficient
An early (three-day!) project involved taking the local little league and building a web site featuring much of the same information available about major league teams. The kusports.com sites allows subscribers to sign up for baseball stats as short messages tot heir cellphones. Many other examples provided convincing evidence that Django is indeed versatile.
Last year (at PyCon 2005, when Djanfo was about 18 months old) Adrian gave a presentation on what was then "the CMS", and received a lot of feedback that suggested an open source distribution would be popular. Django is built around:
A URL dispatcher
A database wrapper
A template system
An admin framework, and
"Other pain relievers".
Django URLs are designed to be pleasant and readable. The scheme allows the programmer to write Python code that directly maps the URLs to program callables. This decouples the URL from the business logic. The callables, known as "views" in Django terminology, take a request object as their first argument (possibly with values extracted from the URL as additional arguments) and returns and HttpResponse object.
The usual development model is to design the database. Django helps you to create and populate the tables (though it doesn't stop you from using SQL if that turns out to be easier). The design process also results in the automatic generation of admin functionality that can be used imeediately by editorial staff to manage content. The Django team is fortunate to have a designer to build great interface pages.
The initial design is now production-ready, and the team can write the views and templates to produce the required pages. The templating language was deliberately kept simple (avoiding XML and programming languages) to maximize its utility to the designers.
The framework is fully ready for internationalization (the first add-on contributed to the project after it was open-sourced), and provides three different levels of caching: per-site, per-view and low-level (this latter being performed in raw Python).
Repeatedly-required view functionalities have been abstracted into "generic views" that are automatically available to developers with no further formality. Sites using Django include chicagocrime.org (an award-winning site that integrates with Google maps), washingtonpost.com (many feeds are provided by screen-scraping Python scripts), the Ellington system (an online publishing system for news and entertainment sites) and traincheck, a site you can SMS to find out when and where to catch your metro.
The 1.0 release is approaching, with much work upcoming in a post-Python sprint, and a book is expected later in the year from APress.
A nice final touch was the announcement of the availability of t-shirts featuring the slogan
I wish I were
powered by
Django
It was great to see enthusiastic and capable developers using Python for pragmatic purposes. Django does not attempt to be a fully-integrated content-management framework, but a loosely-coupled collection of components that work extremely effectively together to provide extremely efficient mechanisms for buildding database-driven web sites. Go to djangoproject.com and take a look!
The State of Dabo
Ed Leafe is an engaging speaker, and the principal developer of Dabo, a desktop application development tool (which, as he says, is a novelty nowadays).
Design goals for Dabo included database independence, UI independence (they chose wxPython as their base for its clean look, maturity and portability) and a simplified API. The GUI development aspect of Dabo has led to rapid takeup by wxPython users without database needs, but the primary target is the world of developers with Visual Studio or Visual Basic experience. These people need to see attractive development platforms with similar capability levels before they will feel comfortable migrating away from the proprietary world.
Since last year Dabo has added a report designer that enables visual design; the same principles are being applied to class design, and there's a new editor. Ed gave a quick demonstration of application building, explaining that Dabo assumes you already have a data repository, and that you want to run applications against it.
The report writer creates PDF files from an XML report description (and some data, of course). The report designer isn't (yet) hugely sophisticated, but it's better than much of the competition in the open source space. Like the rest of Dabo it''s cross-platform. The GUI editor has come on immensely in a year, and is capable of producing versatile layouts without the heavy lifting required by wxPython when used directly.
Dabo doesn't use any object-relational mapping layer, as the developers are "traditionalists".
A nice presentation of a solid project that's obviously continued to receive a lot of attention from its developers in the last year.
Design goals for Dabo included database independence, UI independence (they chose wxPython as their base for its clean look, maturity and portability) and a simplified API. The GUI development aspect of Dabo has led to rapid takeup by wxPython users without database needs, but the primary target is the world of developers with Visual Studio or Visual Basic experience. These people need to see attractive development platforms with similar capability levels before they will feel comfortable migrating away from the proprietary world.
Since last year Dabo has added a report designer that enables visual design; the same principles are being applied to class design, and there's a new editor. Ed gave a quick demonstration of application building, explaining that Dabo assumes you already have a data repository, and that you want to run applications against it.
The report writer creates PDF files from an XML report description (and some data, of course). The report designer isn't (yet) hugely sophisticated, but it's better than much of the competition in the open source space. Like the rest of Dabo it''s cross-platform. The GUI editor has come on immensely in a year, and is capable of producing versatile layouts without the heavy lifting required by wxPython when used directly.
Dabo doesn't use any object-relational mapping layer, as the developers are "traditionalists".
A nice presentation of a solid project that's obviously continued to receive a lot of attention from its developers in the last year.
Impressions from "Python for S60" by Matt Croydon
Jurgen Schieble has wriiten an awesome tutorial.
PyS60 provides an application user interface layer, which applications can leverage to reduce code size and easily interact with users, as the appuifw module. The development environment makes it really easy to test code on a real device rather than in an emulator.
While the whole standard library isn't present a useful set is provided, and many pure Python standard library modules can be added and will run wihtout change.
Specific libraries offer telephone-based functionality such as
The PDIS team have added many libraries, and Nick Burch has built a webbrowser.py.
Earlier applications tended to be quick hacks, but the more recent batch of applications are really polished. Traffic cam viewing, stock quote grabbing dictionary lookup and weather display were among those mentioned.
Installation sounds really easy.
The S60 world is excited about Nokia's recent release of the Python implementation as open source, and applications are rapidly getting more mature, with an active community pushing the technology forward. Lots of sample code is available online.
Slides for the talk are available at http://postneo.com/talks/pycon2006/
PyS60 provides an application user interface layer, which applications can leverage to reduce code size and easily interact with users, as the appuifw module. The development environment makes it really easy to test code on a real device rather than in an emulator.
While the whole standard library isn't present a useful set is provided, and many pure Python standard library modules can be added and will run wihtout change.
Specific libraries offer telephone-based functionality such as
image = camera.take.photo()You can manipulate your contacts and calendar easily from Python.
telephone.dial('411')
The PDIS team have added many libraries, and Nick Burch has built a webbrowser.py.
Earlier applications tended to be quick hacks, but the more recent batch of applications are really polished. Traffic cam viewing, stock quote grabbing dictionary lookup and weather display were among those mentioned.
Installation sounds really easy.
The S60 world is excited about Nokia's recent release of the Python implementation as open source, and applications are rapidly getting more mature, with an active community pushing the technology forward. Lots of sample code is available online.
Slides for the talk are available at http://postneo.com/talks/pycon2006/
Some amusing and interesting ideas from the keynote "Plone: It Ain't About the Software" by Alexander Limi and Alan Runyan.
The $4 million company that operates by emailing Excel spreadsheets around. Not intranet, just a single shared drive where one user can delete another user's files to make space for their own.
The idea of the "J2EE Liberation Front". Those guys need a serious dose of freedom. [I gave up Java a long time ago because it seemed too like a straight jacket].
The Plone world has taken translation issues seriously, with the base system now available in 50 different languages. This issue isn't yet taken seriously in America, but the more dependent the US economy becomes on foreign markets the more attention it will get.
They claim that Plone was easy to install, so I shall be testing that assertion. It's a long time since I played with Plone (back in the days when the Python Business Forum decided to use it as their main web presence), and it sounds like it's a completely different deal now.
I really enjoyed the story about the company that had bought an in-house training class, and allocated the two spare seats to members of their janitorial staff. At the end of the course the cleaners were happily writing Python!
I think the speakers started out nervous, but they hit their stride halfway through (PyCon audinces are pretty friendly) and made some interesting points about how frameworks by themselves have little value. What people pay for is a tool that will help them communicate their ideas more successfully. They were realistic about Plone, admitting that the insides were ugly in places, but that from the outside Plone is a usable solutions for many content management needs.
Q: "What's new in the upcoming 2.5 release, and are users demanding AJAX functionality?"
A: Introduction of more Zope3 components as part of a gradual migration. AJAX will be a 3.0 feature. The developers aren't yet decided on the kind of AJAX framework they will use. Sites with usability concerns, such as government departments, are unlikely to use AJAX anyway.
End of the keynote. There are signs that the wireless network is under stress (hardly surprising with the PyCon audience). STSN, the wireless provider, have never had to cope with this density of demand at the venue, so they have sensibly put a couple of staff on-site to learn how their infrastructure reacts to this load and provide help as necessary. Still, I can get through to Blogger to post, and despite occasionaly dropouts I am receiving email, so I'm pretty sure that connectivity will be OK.
Next for me (after coffee) is Python in Your Pocket: Python for the Series 60. Last year's talk by Erik Smartt was one of the many I had to miss. Since the BDFL now claims to have a Python program answering his cellphone maybe I'll have to look at a change of technology myself.
The $4 million company that operates by emailing Excel spreadsheets around. Not intranet, just a single shared drive where one user can delete another user's files to make space for their own.
The idea of the "J2EE Liberation Front". Those guys need a serious dose of freedom. [I gave up Java a long time ago because it seemed too like a straight jacket].
The Plone world has taken translation issues seriously, with the base system now available in 50 different languages. This issue isn't yet taken seriously in America, but the more dependent the US economy becomes on foreign markets the more attention it will get.
They claim that Plone was easy to install, so I shall be testing that assertion. It's a long time since I played with Plone (back in the days when the Python Business Forum decided to use it as their main web presence), and it sounds like it's a completely different deal now.
I really enjoyed the story about the company that had bought an in-house training class, and allocated the two spare seats to members of their janitorial staff. At the end of the course the cleaners were happily writing Python!
I think the speakers started out nervous, but they hit their stride halfway through (PyCon audinces are pretty friendly) and made some interesting points about how frameworks by themselves have little value. What people pay for is a tool that will help them communicate their ideas more successfully. They were realistic about Plone, admitting that the insides were ugly in places, but that from the outside Plone is a usable solutions for many content management needs.
Q: "What's new in the upcoming 2.5 release, and are users demanding AJAX functionality?"
A: Introduction of more Zope3 components as part of a gradual migration. AJAX will be a 3.0 feature. The developers aren't yet decided on the kind of AJAX framework they will use. Sites with usability concerns, such as government departments, are unlikely to use AJAX anyway.
End of the keynote. There are signs that the wireless network is under stress (hardly surprising with the PyCon audience). STSN, the wireless provider, have never had to cope with this density of demand at the venue, so they have sensibly put a couple of staff on-site to learn how their infrastructure reacts to this load and provide help as necessary. Still, I can get through to Blogger to post, and despite occasionaly dropouts I am receiving email, so I'm pretty sure that connectivity will be OK.
Next for me (after coffee) is Python in Your Pocket: Python for the Series 60. Last year's talk by Erik Smartt was one of the many I had to miss. Since the BDFL now claims to have a Python program answering his cellphone maybe I'll have to look at a change of technology myself.
PyCon Has Started!
Well, the tutorials are over, for better or worse. If anyone who attended either (or both) of "An Introduction to wxPython" or "Using Databases with Python" happens to read this I'd be very interested in your feedback. There was no formal process for capturing attendee opinions, so I'd really like to know what went well and what could be improved.
During the runup to the conference there was some debate about whether tutorials would be worth organizing, but I think the attendance figures have left little doubt that the community's opinion is in favor.
As I sit here I'm listening to the PyCon 2006 keynote by Alexander Limi and Alan Runyon, and enjoying the rather strange experience of not having to do anything to help with the organization. After three years it's a huge change to be able to attend sessions and just chew the fat with people I meet every year. Signs are that Andrew Kuchling and the support team local Texas user group have done an excellent job.
More later, when the keynote's over.
During the runup to the conference there was some debate about whether tutorials would be worth organizing, but I think the attendance figures have left little doubt that the community's opinion is in favor.
As I sit here I'm listening to the PyCon 2006 keynote by Alexander Limi and Alan Runyon, and enjoying the rather strange experience of not having to do anything to help with the organization. After three years it's a huge change to be able to attend sessions and just chew the fat with people I meet every year. Signs are that Andrew Kuchling and the support team local Texas user group have done an excellent job.
More later, when the keynote's over.
February 22, 2006
PyCon is Here!
After (yet another) horrendous journey (I'm starting to think the airlines have a personal grudge against me) I have managed to get to the Addison Marriott Quorum in time to get dinner and have a good night's sleep before I present my tutorials on databases in Python and wxPython.
Since this is the first year I won't have been chairman it'll be interesting to attend as a regular delegate, and get to go to more of the presentations.
Since this is the first year I won't have been chairman it'll be interesting to attend as a regular delegate, and get to go to more of the presentations.
A Very Poor Map Indeed
Talk about not being able to see the wood for the trees ... I still don't know where Addison is in relation to Dalas and Fort Worth. So I'm off to (where else) Google Maps to find out!
February 11, 2006
Polite Incompetence
I occasionally keep in touch with various non-technical blogs (if only to pretend to myself that I have a life outside of technology). One such blog is that of Seth Godin, who appears to concern himself professionally with marketing. Seth's current latest post is to be found at
http://sethgodin.typepad.com/seths_blog/2006/02/hearing_listeni.html
and said piece ends with a heartfelt question:
This puts me in mind of my erstwhile neighbor Simon Hacker's theory of "polite incompetence". The US (and specifically the Virginia) employment market is currently so under-supplied that employers find it profitable to employ someone who clearly does not possess the skills to provide the goods or services that the employer contracts to provide. As long as the incompetent employees are polite enough, and apologetic enough about not being able to give the necessary service, that they avoid giving offense to the customer their employers, incredibly, manage to stay ahead of the game.
Where once we would have voted with our feet (not to mention our wallets) and gone to some other supplier, we now apparently choose to suffer the slings and arrows of outrageous (but inoffensive) incompetence rather than tell the people we deal with that they clearly do not know what they are talking about and take our business elsewhere.
Adam Smith must be turning in his grave.
I am happy to say that there are still bargains to be had if one bothers to look, and among these I count the Complete Care contract for my Dell Inspiron 9200. The system is now just over a year old and until a month or so ago had been operating without missing a beat. Lately it has developed an intermittent fault which, when present, appears to replicate the same scan line across the whole of the screen. The intermittent nature of the fault meant that I have been unable to collaborate with Dell's fault-finding process but they have noentheless already replaced the video card. Since the fault is still present I expect to see another engineer (but this time in Boston: the last fault call was handled in Philadelphia).
Several people have enquired what I paid for this laptop and, when I told them, have expressed the opinion that they could have bought it for less. Which indeed they might for all I know. As far as I'm concerned the Complete Care contract is non-negotiable, and terrific value for money. Without the contract I might have been compelled to part with my system (possibly several times) to allow the Dell support group to repair it. The on-site repair alone is worth the cost of the maintenance contract. Had I not been supported in this way you would not now be able to read this blog (no exaggeration).
So, do you think I'm going to buy some Brand X laptop next time around? I seriously doubt it. Thank you, Dell! See you when I need a new laptop (and hopefully not before :-)
http://sethgodin.typepad.com/seths_blog/2006/02/hearing_listeni.html
and said piece ends with a heartfelt question:
The folks at Eata Pita don't get the Hearing, Listening, Action scenario.Well, there's a good reason why they don't: it's a lot cheaper to keep the payroll down and provice a crappy service than it is to take the effort to supply your customers with what they actually want. Most consumers are so focused on achieving the lowest possible cost of acquisition that they don't critically evaluate whether what their money buys them is what they really want or need. The remaining five percent are vocal about what's wrong with what they have received, but only after they have received (and therefore paid for) it and are thus stranded without leverage.
You get it, it's obvious. So why don't they?
This puts me in mind of my erstwhile neighbor Simon Hacker's theory of "polite incompetence". The US (and specifically the Virginia) employment market is currently so under-supplied that employers find it profitable to employ someone who clearly does not possess the skills to provide the goods or services that the employer contracts to provide. As long as the incompetent employees are polite enough, and apologetic enough about not being able to give the necessary service, that they avoid giving offense to the customer their employers, incredibly, manage to stay ahead of the game.
Where once we would have voted with our feet (not to mention our wallets) and gone to some other supplier, we now apparently choose to suffer the slings and arrows of outrageous (but inoffensive) incompetence rather than tell the people we deal with that they clearly do not know what they are talking about and take our business elsewhere.
Adam Smith must be turning in his grave.
I am happy to say that there are still bargains to be had if one bothers to look, and among these I count the Complete Care contract for my Dell Inspiron 9200. The system is now just over a year old and until a month or so ago had been operating without missing a beat. Lately it has developed an intermittent fault which, when present, appears to replicate the same scan line across the whole of the screen. The intermittent nature of the fault meant that I have been unable to collaborate with Dell's fault-finding process but they have noentheless already replaced the video card. Since the fault is still present I expect to see another engineer (but this time in Boston: the last fault call was handled in Philadelphia).
Several people have enquired what I paid for this laptop and, when I told them, have expressed the opinion that they could have bought it for less. Which indeed they might for all I know. As far as I'm concerned the Complete Care contract is non-negotiable, and terrific value for money. Without the contract I might have been compelled to part with my system (possibly several times) to allow the Dell support group to repair it. The on-site repair alone is worth the cost of the maintenance contract. Had I not been supported in this way you would not now be able to read this blog (no exaggeration).
So, do you think I'm going to buy some Brand X laptop next time around? I seriously doubt it. Thank you, Dell! See you when I need a new laptop (and hopefully not before :-)
Subscribe to:
Posts (Atom)