Mark Shuttleworth at OSCON

Mark, an accomplished speaker, started by confessing that he had no doubt that Linux would be the platform of the future, and suggesting that tools like Firefox and bazaar had claimed ownership of their application space by providing extensibility in a modular way: conform to the interfaces, and the application becomes a platform. All the same, he insisted there is also a need to work with Windows (which makes sense since there are so many Windows systems around).

One of the issues Mark discussed was how we can extend agile techniques in ways inspired by community-driven processes. Architecting your tool set and your enterprise for collaboration and communication is essential. Ubuntu's governance is entirely separated from Canonical's, and Mark suspects that might turn out to be a best practice.

Tools must be extensible, and whether they are open source or proprietary they must be usable as components in diverse systems. The ultimate goal is "permissionless" development - new people coming in wanting to do new things need to be able to take the tools and run with them to implement their own ideas. His ideal for Ubuntu is for any developer worldwide to branch the Ubuntu code and publish their version in ways that other people can in turn pick up and run with.

Another best practice Mark has identified is time-based releases, which lend a rhythm to the development process and allow developers to keep track of what's happening in the trunk, which should be releasable at any time. More and more projects are thinking about synchronizing their releases. Imagine if a multitude of Linux distributions could collaborate to release at the same time, providing a "development cadence", encouraging individual tool developers to finalize their own releases for incorporation into a multitude of distributions.

Mark sees todays open source development world as struggling to provide free software, but the questions is how will the economic models support this? Free software is easy to love: it costs nothing to acquire. But who will fund the development of the free software? This will require innovations in economics that will, with a historical perspective, possibly be one of the more significant changes in these turbulent times. Having three almost-monopolies each controlling their own distribution would be counter-productive, and advertising on the desktop is offensive.

At Canonical they are hiring people to work on the desktop to build tools that benefit the providers of services. This will fundamentally change the world.

The challenge of the next two years is to lift the Linux desktop to a level where it is effectively art! How do Linux distributions not emulate Apple, but blow right past them. Jamie Zawinski says we should be aiming to build software tools that help their users get laid! The only doubts Mark had about that as a goal were related to Jamie's stated goal of making software that his mother could use ...

The Web 2.0 explosion has taught us that to survive in the web world a software product needs to be instantly attractive, and deliver functionality immediately in a profoundly usable way. Canonical have been investing in the process and the technology, and they are now trying to
realize the dream that Mark so elegantly elucidated in his presentation.

Django 1.0 Alpha 1 released

We learned yesterday that the Django development team had released the first alpha on the trail to the 1.0 release due early in September.

This is great news: it means that the major items that have been developed on the sidelines (a refactoring of the queryset code and a rewrite of the admin application to use NewForms) are now fully merged into the trunk. Most of the functionality for the eventual release is now in place, though of course the feature freeze doesn't start until the betas, scheduled for two weeks time.

OSCON Bound

Flying out to Portland tomorrow arriving late evening (I only just got back from California, and I wanted a little time at home). I am giving a Python in 3 Hours tutorial on Monday morning, but don't have many other set commitments during the week.

Anyone who wants to get together for a meal or a drink, just leave a comment here, or email me.

CPython Getting Serious About Quality: Help Needed

Beta2 of Python 2.6 and 3.0 was due out today. It isn't going to happen, and for the best of reasons. Under the guidance of release manager Barry Warsaw the developers have started to pay serious attention to the output of the buildbots, the systems which perform continuous integration testing of the development version of the Python interpreter.

Barry has just announced that there are three issue serious enough to delay the release, even though this potentially means slipping the release schedule (though time will tell whether that's a serious concern). If you are a Python developer, please take notice of what he said in an email to the python-dev list earlier today:

Please everyone, if you have only a little bit of time to work on Python, I hope you will attack the release critical and deferred blocker issues, and work on turning the buildbots green.  These are the top priorities in order to get 2.6 and 3.0 out on time.  And just as added incentive, our October 1st goal is being noted by downstream vendors. [Possibly confidential information about third parties redacted].

Quality is important. Please do whatever you can to help. Many people can benefit from a little extra effort now.

PyPy Continues to Catch Up

Interesting news from the EuroPython sprints at Vilnius via Holger Krekel. PyPy supports Django well enough to run the Pinax application put together by an open source team coordinated by James Tauber. It's great to see Python expanding in so many directions at once. The future is definitelyto be interesting.

Django seems to be becoming one of the "how good is your implementation" tests for Python implementations. Not only are the PyPy team using it in this way, but IronPython and Jython are as well. Since Jython is now in alpha with a Python 2.5-compatible implementation the picture is looking better and better every day.

The one thing nobody has discussed much is the attitude of the PyPy, Jython and IronPython development teams to migration towards Python 3.0. Does it represent the next wave, or will it end up as an isolated development?

Shameful Day in American History

So, it's official. Corporations are above the law, individual freedom should be subservient to corporate profits and if the president tells you to break the law well you can just go ahead and break it. The revolution moves a little nearer.

Certificate Design Required

Do you know anyone who can design a really nice looking certificate? Even better, are you such a person? The PSF instituted an awards scheme recently and the first four awards were announced at PyCon earlier this year. I have to have an official certificate created so we can have copies framed for the recipients, and my original designer has let me down (too busy).

So I am asking the web if it knows a suitable designer, preferably coming with recommendations, who can quote me a price (I can provide a reasonable brief if you contact me via this link) and get this job off my plate before the delay becomes too embarrassing.

July 8-10 Course Purchases

Please note: if you have attempted to purchase a place on our class on July 8-10, the purchase button was taking you to a page that showed the original (June) dates for the class. We have now changed this to agree with the current schedule, so you will see the correct dates on the purchase page. Please accept our apologies for any confusion.

No matter which button you used, if you have purchased a place on the class you should shortly receive a confirmation letter for your place on the class.

Help Shoot Down New Telco Immunity Threat

From the Electronic Frontier Foundation [italics added here]

ACTION ALERT: THE SHAM TELECOM IMMUNITY "COMPROMISE" IS LURKING IN THE WINGS, awaiting the signal for showtime. Though some members of Congress are guilty of selling the [rumored] bill's telecom immunity provision as a "compromise," there is still time to expose the telecom immunity proposal as a sham and to halt the bill before its too late. Already, some Congressional leaders have expressed concerns, and other influential members have expressed resistance to the scheme. Please join in and make your voice heard to all the members of Congress -- no false compromises for lawbreaking telecoms!
http://www.eff.org/action/no-false-compromises

I fully expect to see some shameful special-interest pandering by the exiting Bush administration as it breathes its weary last. Corporate America must learn to be responsible for its actions, even when it has broken the law at the President's request. Otherwise it will make foreign nations much more reluctant to house American enterprises. I seem to remember a time when "I was just obeying orders" wasn't accepted as an excuse for any kind of wrongdoing.

Why Twisted? - July 11, Washington, DC

Duncan McGreggor recently announced Holden Web's first public joint venture with Divmod but it struck me that our readerships may be quite disjoint, and so I should mention it too.

I know from Teach Me Twisted that there is a thirst for knowledge about this fascinating system and so I agreed to do something else about that, for a wider audience than PyCon. Consequently Duncan, Glyph and I agreed to follow our next public Introduction to Python class with a half-day seminar on how Twisted can create scalable networking solutions to real problems.

My own contribution will be inspired by Duncan's recent Mantissa: An Alternative to LAMP post, and will attempt to position Twisted in the marketplace. Glyph will be offering some really neat canned demonstrations of what Twisted can do [with a CD for managers to take back to their techies], and I am hoping that Duncan will be able to play the visionary role, taking us all into a future where networking is an essential component of all systems.

So, if you have a need to know about Twisted, and especially if you have budget, reserve a place now. There are only twelve places, and some are already taken.

We will give you two hours of presentations and follow that with a lunch where you can ask the speakers questions at leisure before departing to the remainder of your Friday afternoon. Sign up now!

Seth Excels Himself

Mostly this is a technical blog, but I feed a bit of marketing to the geeks now and then. I keep an eye on Seth Godin's blog, and this post caught my eye today. It ends with

The object isn’t to be perfect. The goal isn’t to hold back until you’ve created something beyond reproach. I believe the opposite is true. Our birthright is to fail and to fail often, but to fail in search of something bigger than we can imagine. To do anything else is to waste it all.

Amen to that. The person who never failed at anything never tried anything worthwhile.

"Introduction to Python" Class: July 8-10

Holden Web is pleased to announce its second public "Introduction to Python" class in the Washington, DC area. Intended for those with some programming background, the class gives you a working knowledge of the Python language reinforced by hands-on practical exercises. The class will be taught by Steve Holden, consultant, trainer and author of Python Web Programming.

If you know anyone who needs to learn Python (and who doesn't?), please point them to the registration page.

Divmod Partnership

Occasional readers may not remember how energized I was after my Teach Me Twisted session at PyCon this year, but I definitely wanted to follow it up with more. Audience members from that session will need no convincing that my ignorance of Twisted was almost total, but I am happy to say that it has now diminished a little. Determined effort on both sides has now yielded a training strategy for teaching other people Twisted too.

Twisted training will benefit from an analysis of the concepts that allows us to present them in the order required for specific understanding by a given client group. Divmod and Holden Web are jointly developing presentations that should make Twisted concepts approachable for a wider audience.

I am really looking forward to getting into some joint projects with the guys at Divmod. They have brains the size of planets. Stay tuned for an announcement about an introductory presentation.

Twisted Stakes a Claim

Disclosure: this post discusses remarks made by a Holden Web business partner.
In his Electric Duncan blog, Duncan McGreggor argues that Twisted now provides a viable LAMP alternative, and asks some very pertinent "so what?"-type questions:

As such, this is a direct competitor for LAMP. Here are some questions about that: What is the value of a full stack? Why is an alternative to LAMP good or needed? What is a good alternative? [...] We need to be asking ourselves what our applications are, what the network is, what services are, and what our dev teams and engineers need.

This led me to ask some questions of my own. Is this true? Is Twisted actually able to provide a full-service alternative web application solution? How adaptable is the architecture? Suppose I want to retain my Django templating instead of transitioning to Nevow: how easy would that be?

The case against LAMP starts with out-of-the-box scalability, as Sean Reifschneider (among other luminaries) has pointed out. I believe that Twisted has been proven at reasonably high levels. Whether it's processing Google-sized firehoses I don't know, but it's certainly supporting a significant part of the airline reservations structure.

As to "what is a good alternative", I suspect a lot of developers would go with "something that's orthogonal to my current code, so I don't have to rewrite anything". Perhaps that's asking a bit much, though. I know from various transitions I have made that some things just "fit my brain", and one of those was the Django templating scheme. Django users are pretty much database-agnostic, though a surprising number of them seem to use PostgreSQL. So perhaps Axiom support of Django templating?

The thing is, you can't just go around claiming to be an effective competitor to LAMP. You have to demonstrate the truth of that claim. I am sure the Twisted guys can do that if they are forced. They just don't realize the marketing necessity yet.

Nicholas Negroponte: An Ego Unbound?

I just finished reading Ivan Krstic's excellent Sic Transit Gloria Laptopi blog post. It appears that Negroponte would now like the OLPC XO to become yet another laptop we have to throw Windows off before we install some open source operating system supporting the GNU toolset.

I don't believe Nicholas Negroponte could manage to find his way out of a paper bag, let alone manage a hardware supplier trusted to build the infrastructure of developing countries. His Windows bait-and-switch makes him look like a shill for Microsoft. I wonder how much Microsoft stock he holds?

He just doesn't get that he has blown a huge chance to make a difference here. The thing that pisses me is the way he has played fast and loose with the work of literally hundreds of open source programmers who have attempted to support what they saw as a worthwhile educational project. But now Negroponte says that OLPC isn't an educational project after all. It's not about you. Nick, it's about the kids in the developing economies.

Thanks for the Money!

Well, apparently seven thousand other people also wanted to support brain tumor research yesterday at the eleventh annual "DC Race for Hope" in aid of the Brain Tumor Society and Accelerate Brain Cancer Cure. So many, in fact, that I didn't even pass the starting line until ten minutes after the first contestants. But it wasn't all about speed, which is just as well, since the lead runner past me on his return journey before I had even covered the first mile of the course.

The most important thing was the huge amount of money the event generated (a donation of a million dollars was announced just before the race commenced). Thanks once again to everyone who helped me to smash my sponsorship goal. With your help I raised $1,215, handsomely exceeding my target of $1,000, and my wildest expectations.

For those of you kind enough to inquire about my friend Sharon, her tumor turned out to be treatable with a combination of surgery and other therapies, and with luck she will be recovering from the surgery as I write this. I'm hoping that next year she can come to the race and join the other survivors. Thanks again for all your help!

The Media We Deserve?

Many thanks to Patrick Logan for bringing this New York Times article to my attention. It highlights the fact that the "military analysts" used (and paid!) by the major networks often appear to have business dealings and Pentagon relationships that cause blatant conflicts of interest with their independence as analysts.

My favorite quote shows that the networks feel they have something to hide, and in particular just exactly how "fair and balanced" Fox News wants to be:

CBS News declined to comment on what it knew about its military analysts’ business affiliations or what steps it took to guard against potential conflicts.

NBC News also declined to discuss its procedures for hiring and monitoring military analysts. The network issued a short statement: “We have clear policies in place to assure that the people who appear on our air have been appropriately vetted and that nothing in their profile would lead to even a perception of a conflict of interest.”

Jeffrey W. Schneider, a spokesman for ABC, said that while the network’s military consultants were not held to the same ethical rules as its full-time journalists, they were expected to keep the network informed about any outside business entanglements. “We make it clear to them we expect them to keep us closely apprised,” he said.

A spokeswoman for Fox News said executives “refused to participate” in this article.

CNN requires its military analysts to disclose in writing all outside sources of income. But like the other networks, it does not provide its military analysts with the kind of written, specific ethical guidelines it gives its full-time employees for avoiding real or apparent conflicts of interest.

Nothing like shutting up when you have something to hide. You can almost hear the ranks closing.

Microsoft Can't Support DRM

So it seems that "PlaysForSure" actually meant "Plays until the first time you change your computer or operating system after August 31, 2008". About ten days ago Microsoft's general manager for MSN Entertainment and Video Services sent out an email to customers advising them that license keys can no longer be retrieved and new computers cannot be activated after that date.

This is a very clear indication that digital rights management (DRM) is a loss for the consumer. If I wanted to keep songs from my eight-track stereo around then there was nothing to stop me from transferring them to audio cassette or CD (despite the RIAA's rather specious arguments that this was a breach of copyright). If I'd been silly enough to trust Microsoft's assertion that DRM offered me protections then I'd have to get ready to shell out for new copies after any change to my playing environment.

The Electronic Freedom Foundation has written to Microsoft stating in no uncertain terms that (by suggesting that users copy the music to a CD and transfer it to new computers that way) the company "is asking its customers to invest more time, labor and money in order to continue to enjoy the music for which they have already paid".

If Microsoft, with all their resources, can't keep a DRM scheme working then there is little chance for anyone else. It also doesn't encourage customers to trust the company if it is prepared to abandon them in this way, though they might have anticipated this desertion when Microsoft closed the MSN store when it started selling the Zune.

I suspect this is the beginning of the end of DRM. Let it rest in peace.

Twisty Little Passages ...

I have been rather disappointed that the "Teach Me Twisted" session I ran at PyCon didn't get more heavily blogged. Not only did I have a lot of fun, but I also learned something. The Twisted guys learned something too, I'm happy to say.

No announcements yet, but with luck it will eventually lead to bigger things. That session may have been a butterfly flapping its wings.

Django Nudges Closer to 1.0

Malcom Tredinnick just announced that he has merged the long-awaited queryset-refactor branch into the Django trunk. This puts one of the major blocks in place for the 1.0 release.

James Bennett recently posted his list of requirements for a 1.0 release (with a warning that suggestions from twits would not be tolerated: do not annoy this man if you don't know what you are talking about), and this announcement reduces the length of the list to eleven items.

So it's got significantly closer, as queryset-refactor was a biggie. Now I have to go test my code against the new branch ...

Impressive Silly Trick

Nice to see that the Python community still enjoys a challenge!

I Want Your Money!

Not for myself, I hasten to add, but to support the Brain Tumor Society. The Race for Hope 5km run is an annual event organized by my wife's employer, and we had agreed to participate before we discovered recently that one of our friends has a brain tumor. It's too early to know whether it's operable at this stage, so we are hoping for the best.

[UPDATE 4/22: I have discovered that my wife has already raised almost $2,500 so I have upped my target somewhat to see if we can't show the world that geeks aren't cheap!]

In the meantime, I have set what I feel is a fairly modest goal for additional sponsorship. I am hoping readers will show how generous the open source world can be. If you've ever enjoyed a PyCon then please drop by my personal race page to make a donation, and point your friends to it as well. I'll be quite happy to up the target if the existing one proves too easy, and any amount will be welcome.

Thanks to anyone generous enough to support this effort. If the team I'm on is in the top 5 fund raisers next week we'll have an extra $1,000 added to our sponsorship, so dig deep and treat the target as something to beat, not exceed.

Impeach?

I mostly keep politics (except perhaps the politic of open source) out of this blog, so if you aren't interested you won't want to follow this link. I'm not sure why I got so fired up about this today, specifically, except that the day's news has been full of politics. But when wasn't it, lately?

Blogger Annoyances #319

Is this just me, or is there some trick I just haven't learned? I was reminded, yet again, when completing the previous post in this blog, that Blogger tagging is way less user-friendly and convenient than del.icio.us and other similar sites.

I wanted to tag the item with "mono", but alas I was foolish enough to tag a previous post with "monopoly". So when I enter "mono" the widget is already suggesting the auto-completion (see image), and when I type a comma, it completes the entry as "monopoly", the existing tag. I try entering an escape character—no good, the comma still completes it as the existing tag, as does a tab.



Eventually I let it complete as "monopoly" and then go back and delete the last four characters from the tag. I really dislike applications that insist they know better than me. Since tab expands the tag to complete as an existing tag, why the heck does a comma have to do the same thing? The real annoyance is: I have this ugly feeling that somewhere out there there's a programmer feeling really smart for having committed this atrocity. Someone, please tell me there's a sensible way to do this before I go mad.

Learning IronPython

Some good advice from Curt Hagenlocher on the IronPython mailing list in response to this question:

I program in Python, but my programs are typically run from the command line or IDLE. I have never built a GUI, and would like to try my hand at creating GUIs that I can run my programs from. Is there a book or tutorial that covers IronPython, IronPython Studio and the dot Net platform that is aimed squarely at the new user?

Any book or tutorial about the .NET platform in general will be relevant to IronPython's use of .NET. In fact, taking existing C# or VB examples and translating them into Python may be a good way to leverage your existing Python skills while teaching yourself the .NET class libraries.

As far as IronPython-specific resources go, you can't beat the IronPython Cookbook at http://www.ironpython.info/index.php/Main_Page. If you want to see some sample Python code that uses the Windows Forms or Windows Presentation Foundation libraries to produce a GUI, just go to the Contents at http://www.ironpython.info/index.php/Contents and scroll down a bit.

It's fashionable to pooh-pooh .NET, but remember that this stuff runs on Mono too, so it could be an important way to build cross-platform compatibility in Python applications. Not everything that came out of Redmond is worth ignoring (but let's not talk about OOXML), so there's possibly something to be gained from a knowledge of that framework. Don't throw the baby out with the bathwater.

Galcon Cheats!

I'm not going to ask for my money back, or anything, but I really do think it's a bit naughty of Galcon to mobilize its forces before I've even pressed the start button.

If you haven't tried this game by Phil Hassey (met him at PyCon, terrific bloke) it's quite an addictive space war adventure where the objective is to overrun neutral and opposing forces to gain control of planetary territory.

You can also run it as a network server if you want to host multi-player games! Well worth a try. But it still cheats, dammit.

For the Want of a Nail

We've all done it. One step forward, two steps back. I just thought I'd try to relieve my current frustrations by writing this particular sequence up. It's amazing the diversions and byroads you can end up taking to achieve an apparently simple goal (as I learned yet again last week in trying to accommodate Outlook on my SMTP server). This one started at PyCon, now four weeks in the past. So, bear in mind the goal: submit a Django documentation change.

At the Django sprint (of which, due to a family illness, I was only able to attend a truncated first day) I was struggling with an unstable laptop, so I managed to assign myself a documentation ticket, and that turned out to be pretty much it what with PSF responsibilities and needing to go home early. I have been working on the ticket since: it isn't a difficult issue, but I needed to learn a little more about how the Django database layers worked.

Having finally decided on the required change, during entry in the Django tracker I decided it would be much more developer-friendly if I were to provide the change in the form of a patch. This isn't really necessary but I figured it couldn't hurt, and that I should really be able to rebuild the documentation locally to be sure that any future changes were stylistically correct and didn't mangle what is, after all, quite good documentation.

This is where things started to get a little rough. Descending into the documentation directory I discover that it's driven by a Makefile. "No problem", I think, "I have Cygwin, I can build under that". So I type "make html" and discover I don't have make installed. This is quite normal, I like to keep Cygwin fairly lean, and I haven't previously done any Cygwin building on the new laptop.

So, I run the usually very cooperative Cygwin installation utility, and discover that my usual mirror site is providing very slow response. So slow, in fact [minor pause here to repair cable retraction mechanism on traveling mouse: this really is not my day] that several attempts to update my installation have to be abandoned, though it notes that several of my other installed applications should also be upgraded and has even managed to pull down some of the installers.

Eventually the fifth or sixth attempt gives me the dread message from Vista (how I hate this operating system, yet despite my best efforts there are tasks I currently still need it for) that the Cygwin installer "is not working correctly".

After a reboot and several further repeats, I eventually track this down to a corrupted /etc/setup/openshh.lst.gz file, and delete it. I establish contact with a different mirror and lo! The Cygwin installer now runs to completion, and I can run make. So I confidently re-enter "make html" and see the message

make: sphinx-build: Command not found

Fair enough, it shouldn't (I foolishly think, despite all the evidence that should by now have convinced me to set the computer aside for the day) take me long to get that up. As a first stab, before doing anything else, I type "easy_install sphinx-build". Hello, another even more interesting error message:

bash: /c/Python25/Scripts/easy_install: Permission denied

What? ls -l gives me the following rather unsavory news:

----------+ 1 sholden None 6656 Apr 2 10:33 /c/Python25/Scripts/easy_install

Damn, that can't be right? How did that happen? Well by this time I am past caring how it happened. I own the file, so I'll just reassert the correct permission, right? Wrong:

$ chmod 755 /c/Python25/Scripts/easy_install chmod: changing permissions of `/c/Python25/Scripts/easy_install': Permission denied

Now this is getting really annoying. Note how my problem has subtly changed from "how do I submit a Django documentation change" to "how do I get Cygwin to allow me to change the permissions on this file", and that I am now several steps further away from my original goal than I appeared to be at that happy moment when I set out to submit the change. Note also the assumption that running easy_install is actually necessary to solve my problem. If this had turned out to be wrong I would probably have shot myself, thereby relieving you of the task of reading this post.

Fortunately I quickly realize that this is the easy_setup from my Windows Python that I am trying to run, and that the solution is to run ez_setup.py with my Cygwin Python. [There are some who doubt the wisdom of doing this to a Python installation, but I am blasé beyond belief about such things]. So I run ez_setup.py, which installs setuptools. There is the minor annoyance that Cygwin refuses to recognize the correct version of easy_install and continues to run the Windows copy, but I solve that by opening a new Cygwin terminal window. Another final false start: the package required isn't called sphinx-build, so easy_setup can't load it. So I take another reasonable guess.

Now, finally, I run easy_install sphinx followed by make html, and in less time than it take to write a blog entry I have a local copy of the documentation that I can patch to my heart's content. Wonderful things, computers. When they do what you want.

Tweaking the Bytecode

Ned Batchelder has come up with an interesting hack for increasing the granularity of his code coverage testing. It involves modifying the code object associated with the code being modified: specifically, the co_lnotab member contains a starting line number followed by a sequence of (code offset, line number delta) pairs.

Ned's technique involves rewriting the member so that each bytecode appears to be on a single line on its own. While this allows him to ensure that all bytecodes are exercised, he hasn't yet ironed out the quirks in how to use that to present coverage information (which is hardly surprising, since most Python programmers could care less about the structure of their bytecode).

It seems to me, however, that by mapping the modified co_lnotab back to the original you could at least flag lines whose code isn't 100% covered by your tests, and this is surely valuable information. Ned's already thought of this, and includes the throwaway remark "The original line number information is lost. It would have to be stored off to the side to make this useful" towards the end of his blog entry.

So I believe we can expect coverage.py to start producing even more useful information soon.

Meme du Jour

Interesting. I was quite surprised by this myself.

# history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}' |sort -rn|head
135 svn
134 ls
87 vi
76 postmap
57 cd
45 more
36 ps
35 /etc/init.d/httpd
33 /usr/sbin/postfix
32 grep

This is on my (virtual private) web server, where I've been editing fairly heavily recently.

Outlook is Driving Me Nuts!

[UPDATE: two changes were required. First I had to set broken_sasl_auth_clients = yes in my postfix main.cf, then I had to add login as an available authentication mechanism in dovecot, which postfix uses as an authentication server. Thanks for all pointers and suggestions, some of which came by e-mail. Not quite as stupid and fundamental as I feared. Steve].

OK, I am in serious need of assistance here. I suspect that there must be something really stupid and fundamental that I am overlooking. The problem is that I cannot get Outlook 2003 to authenticate to its SMTP server.

I have tried deleting the account and recreating it (shutting Outlook down in between). I have verified that the same settings in a Thunderbird client allow me to send mail without difficulty. I have even installed Outlook on an entirely separate computer and got the same dismal results.

Why Outlook? Well, although I am a Thunderbird user myself my wife uses Outlook. So my most important support client has a problem -- only with sending mail, though, receiving works fine.

Just so you can visually verify that I haven't done anything stupid, here are screen shots of the relevant dialogs. I have even set up a local printing SMTP proxy to verify what's happening! With Thunderbird it clearly shows an AUTH command being send to the server, but with good 'ole Outlook I get what you see below (slightly redacted): straight from EHLO to MAIL with no AUTH at all.

Connected
==> (0) '220 holdenwe.user.openhosting.com ESMTP Postfix\r'
<== (0) 'EHLO DoroPC\r' ==> (0) '250-holdenwe.user.openhosting.com\r'
==> (0) '250-PIPELINING\r'
==> (0) '250-SIZE 10240000\r'
==> (0) '250-VRFY\r'
==> (0) '250-ETRN\r'
==> (0) '250-AUTH PLAIN\r'
==> (0) '250-ENHANCEDSTATUSCODES\r'
==> (0) '250-8BITMIME\r'
==> (0) '250 DSN\r'
<== (0) 'MAIL FROM: <xxxx@holdenweb.com>\r'
==> (0) '250 2.1.0 Ok\r'
<== (0) 'RCPT TO: <xxxx@gmail.com>\r'
==> (0) '554 5.7.1 <xxxx@gmail.com>: Relay access denied\r'
<== (0) 'QUIT\r' 

So, if you can tell me what's wrong you may rescue my sanity! Help me, please, this is driving me nuts.

Google Busts the Web Wide Open - In Python!

Well, if ever there was a compelling reason for using Python it's probably Google App Engine. Here's the canonical "hello world" Google App:

print 'Content-Type: text/plain'
print ''
print 'Hello, world!'

Doesn't get a lot simpler than that, does it?

There's a lot behind this announcement, including a detailed description of the data store (which interestingly seems to bear the same kind of relationship to a classical relational database that Python does to statically-typed languages), access to some of the attributes of authenticated Google account users, the ability to fetch remote URLs and a startlingly straightforward way to send email.

If I were going to write another book about Python web programming I wouldn't be able to ignore this development. What a tempting idea ...

Twisted Bite the Web Bullet

The Twisted community has decided it's time to clean up its web act. The problem for a while has been that there were three "competing" packages that might have formed the basis of any web application: twisted.web, nevow and twisted.web2.

Now they have decided that twisted.web is the right namespace, and the immediate plan is to backport the stable functionality from web2 and the nevow application server into twisted.web, giving a more advanced set of web functionality in a way that offers best compatibility to existing users. Nevow will intially remain, as a pure templating solution, but there's a possibility that functionality might also be repositioned eventually.

This has the smell of a compromise, but as compromises go (and as you'd expect in the Twisted world) it seems to be a pragmatic engineering solution to the current rather confusing situation.

One-Day Class Topics

Since the first Introduction to Python class seemed to go quite well I am thinking about expanding the offerings by slotting in one-day introductory classes on Python-related topics.

From PyCon I know there is a lot of interest in both Twisted and Django. But I wonder how to contact the people who want to learn these technologies. Are they already members of the Python community (to whom this blog is primarily addressed most of the time), or do I need to generate some new marketing channel to reach them?

What other Python technologies do people need a fast introduction to?

Just Because I Can?

Seth Godin as usual makes some interesting points in a blog entry today, including two very salient ones for me:

a) blogging doesn't have to be perfect, so blog and move on rather than waiting for the perfect way to express a transient thought;
b) you don't express any ideas at all by not writing, so writing a little and often may well be more useful than producing large well-crafted essays every six months.

The first is a message that has started to get through to me anyway as a result of my work at Python Magazine; the second is a variation on the theme that nobody knows what history will make of them, so there's little point trying for a particular effect.

Thanks for getting me started again Seth!

Damn Those AdWords!

OK, this is a guilt trip. You have been warned.

One of the things about being an AdWords user is that you realize the cost of a click-through.

So just now, trying to find out exactly how far the proposed $102,000,000,000 (that's an obscene waste of taxpayer dollars) spend on the Iraq war would go towards making sure everyone in the world had potable water to drink, I clicked on a sponsored link. As soon as the page came up I realized that I had just cost these people money.

So, I could have sent them a couple of bucks to salve my conscience. Instead I decided I would give them something scarcer, so I am taking a few minutes of my time to ask you to visit their Web site, and understand (or at least struggle to understand) why one sixth of the world's population is unable to take clean drinking water for granted, and think about what that must be like for them. I figure that if only ten of you click the link then I have probably contributed the equivalent of at least $10. If you bother to post a comment here saying you have clicked, I will also send PlayPumps $6 per person, thereby providing someone with clean water for ten years—I doubt I have so many readers, but prudence cautions me to limit the total to $480.

Then go to my other blog and look at the mounting cost of the Iraqi suppression at the top right hand corner. If the USA spent one-tenth as much providing drinking water to the world as it does spilling blood it would be hailed as a benefactor, and welcomed throughout international society. Wouldn't it be nice to have a few other countries on our side for a change? How about a "coalition of the grateful"?

How come lame duck presidents never seem to grapple with this as they piss away their few remaining months in power? God forbid it should get in the way of the plans for yet another self-congratulatory presidential library. Don't they realize these kids will be programming in Python once they get their OLPCs?

[Final update, April 7: Not as ruinous as feared, but I have
just send PlayPumps a check for $120. Thanks for participating! Steve]

Ye Olde Blog Spam

For the past two days now both Planet Python and Unofficial Planet Python appear to have been receiving about four copies a day of a blog, amusingly (for someone so apparently clueless) entitled Ye Olde Clue.

Could someone please hit the author with ye olde clue sticke and stop this nonsense? We are all used to seeing occasional glitches, but this blog just seems to go over the top.

Goodbye, Arthur C. Clarke

Since my blog is emblazoned with a quotation from the man it would be ungracious of me to let his death last week pass by without some remark. A man of formidable vision, Clarke envisaged the geosynchronous satellite years before anyone else, but was denied a patent because his disclosure of the idea in a Wireless World article was regarded as prior art. (How ridiculous this makes the US Patent Office's current lax diligence in the software field).

He was an inventive science fiction author, and one whose books would frequently withstand the test of a second and subsequent reading. Many were more prolific in volume, but few could challenge him on the scope of his ideas. In later life his move to Sri Lanka kept him in touch with the needs of the people increasingly crammed together on this small shared ball of rock.

Who knows where he ranges now. The world is a smaller place for his passing.

Image: Arthur C. Clarke Foundation

Packaging Python

There's an extensive discussion at the moment on the python-dev list about how python modules and applications should be packaged up in future. This all relates to the distutils and setuptools features, which are great when they work and horrible when they don't. There seems to be general agreement that an uninstall feature would be useful.

My own point of view is that there are different requirements for installing applications for end users and modules for developers. In the former case I believe it would nowadays be easiest and least error-prone if each (Python) application were installed with its own interpreter and all dependencies. While this seems horribly wasteful, it neatly avoids any kind of version lock-in and allows application upgrades. In the latter case then something more like the traditional setup.py install is more appropriate, and the installation probably doesn't need to be recorded in whatever mechanism the system uses to record installed applications (e.g. Add/Remove Programs under Windows).

Whatever the result, it does appear that 2.6 could get us closer to a universal solution to installation problems.

Bruce Perens Needs Your Help

A co-founder of the Open Source Initiative, Bruce Perens is now concerned that the body is becoming increasingly dominated by vendor representatives who do not necessarily have the interests of the open source community at heart.

You can read about why he feels this is necessary on his campaign page. I would suggest that perhaps a change in bylaws is also required, since it appears from Article V Section 5 that the Board of the OSI is, or has the potential to become, a self-perpetuating cadre with no outside accountability. I can't help feeling that's a little unhealthy.

Please visit Bruce's Web site and support him in his campaign to keep Open Source open!

PyCon Friday: That's All, Folks

That's enough blogging, now I'm just going to try and enjoy the conference as much as I can.

PyCon Friday: Python Software Foundation News

Well, the PSF meeting is over, and we have eight new members. The Board of Directors has changed composition slightly, with the departure of Andrew Kuchling (left) and David Ascher (right).

Andrew was the first recipient of the Frank Willison Award for services to the Python community. He did amazing things for PyCon as its second chair (last year and the year before), and has put in huge amounts of unseen work on other issues but unfortunately is now too busy at work to continue as a PSF Director.

David was a founding Director, I believe, and the co-author of early editions of Learning Python and co-editor (with Alex Martelli) of the Python Cookbook. Our loss is the Mozilla Foundation's gain. His new job as head of the corporation to nurture the Thunderbird software meant that he too had to move on, but he's promised to continue to do all he can to support the Foundation even though his new role will clearly be taxing. We all wish him well.

Raymond Hettinger and James Tauber, both long-term supporters and users of Python, now join us on the Board. Raymond may be known to you as the author of the itertools module among his many other fine contributions to the Python core; without his influence Python would still lack a decimal module. James has more than ably represented the Foundation as its Summor Of Code coordinator, and is also a prolific author of code, most recently in the Django arena.

The eight new elected members are: Titus Brown, Mark Dickinson, Amaury Forgeot d'Arc, Christian Hiemes, Van Lindberg , David Mertz, John Pinner and James Tauber. Congratulations to them all. Three new sponsor members were also elected: EarnMyDegree.com, HitFlip and Sun Microsystems. We are happy to have their support.

At this evening's meeting of the newly constituted Board Stephan Deibel followed through on his announced intention to step down after four years as Chairman, though we are pleased to know he is continuing as a Director. I am the new Chairman for the next year. I hope I can do half as good a job as Stephan.

PyCon Friday: Guido's Keynote

Guido started by suggesting that this was going to be boring because it's the same talk as last year: Python 3.0 is coming, it's going to change everything, don't worry!

Matz, the author of Ruby, has said that "open source needs to move or die". Guido sees 3.0 as a chance to remove some of the sticky mistakes he made fairly early in the development of the language. Python is now 18 years old, its first open source release was in February 1991, and it's time for a change.

Guido gave a quick summary of major breakages. Most Python users by now re aware that the print statement is disappearing, that integer division is becoming a separate operator and so on. Guido was more interested in promoting the new features.

He feels that argument annotations will allow users to experiment with the feature, and that some interesting uses will eventually be available in the standard library. Abstract base classes offer an optional way to escape from duct typing. The wild-card unpacking assignment will offer more flexibility, and the new string formatting (similar to C#'s formatting feature), and many small improvements which he just displayed on one single slide.

Oberall Python 3.0 will become a smaller language, and some of the redundant older ways of doing things will be removed in the interests of keeping the language manageable and "fitting your brain". It is ready to evolve, and the new features provide the basis for that evolution. We have a good migration strategy (move to 2.6, then use warnings to make your 2.6 compatible with 2to3 automated translation).

There is no hurry - the 2.x series could be with us for another five years. In the meantime convert to new-style classes, stop using obsolete modules, and make more use of generators and iterators. Leave the small stuff to 2to3, the automated translator.

Porting C extensions may be more troublesome, because many API's will be removed.


Q (Jacob Kaplan-moss): One of Django's features is retaining compatibility with older Python versions, so a full port to 2.6 would effectively break backwards compatibility; what should our migration strategy be?

A: Migrate to 2.6 but don't immediately convert it for migration; track your user base and convert to 3.0-compatible source when it's practical without causing too many problems to your user base.

Q; 2to3 could be useful for frameworks that need to inport other people's code; will it be available with 3.0?

A: It would need some restructuring of the code, but is doesn't sound unreasonable; bring it up on the Python 3000 list, and offer to do a little of the work!

Q: Will the ABCs include an abstract sequence base class?

A: There's a container class, but not something explicit for sequences.

Q: What will the performance differences eventually be between 2.X and 3.0?

A: in the long term I don't anticipate much difference, though 3.0 may take a small hit initially.

Q: You said there would be no change to the whitespace issue - why not?

A: Most of the people here wouldn't let me! But the language is going to be stricter about the potentially damaging uses of mixed tabs and sapces.

Q: Is localization going to start working its way into the way we process strings?

A: I'll have to defer to others on that one; but there won't any longer be a need to work with data in specific encodings.

Q: I was expecting to be able to import 2.6 modules directly into 3.0 as opaque objects, avoiding the need to port legacy code; why can't this be done?

A: A virtual machine that's capable of running both 2.x and 3.0 bytecodes would be a great thing to have, but incompatibilities in the way the basic types operates make this not just ill-advised but impossible.

Q: Is there any change to the GIL strategy for 3.0? Our large applications with Python embedded need to take advantage of multiple CPUs.

A: For better of for worse, it's not going away, and there are good reasons for this, which have started to convince even the most reluctant. The pyprocessing module can help, or look at Jython or IronPython.

Marketing Open Source

Here's an interesting idea. Maybe the PSF should try to fund the development of a stick with Portable Python or Movable Python and a bunch of applications on it? That would have been amazingly useful for the PyCon tutorials this year.

PyCon Friday: the Calm Before the Storm

Late to bed last night, and clearly too sleepy to set the alarm properly, so woke up today feeling a little groggy rather later than I had intended. Laptop behaving so far, fingers crossed.

Breakfast and the keynotes first. For the morning session the only two talks I want to hear clash with each other! PSF members' meeting at lunchtime, then chairing a session in the afternoon (a good way to make sure you get to hear some talks you want to). Trying to leave the evening open for "social business" activities.

I'm asking all the newcomers I meet what made them come to PyCon this year in an attempt to understand the explosive growth. It seems that a lot more people are starting to use Python now, vindicating my ten-year master plan to be involved with a significant technology by the end of this decade. Wonder whether it will do me any good.

Alex Martelli says I should consider writing another book, and wants me to contact a publisher. Whenever anyone says this I always remember the $2.50 royalty check I got once for Python Web Programming, and this brings me back to reality. I told him I might consider it if he'd be my technical editor again. His encyclopedic programming knowledge was terrifically useful last time.

Talking to Glyph Lefkowitz in the bar about books, he asked me how sales were going and I told him it is still in print but I am waiting for it to be remaindered (though it's hardly Books-a-Million material). He was nice enough to say that compared with other six-year-old books on web technologies there was still a lot that was relevant, which is definitely a complement. I did try hard to include explanations of the fundamentals, and I guess that counts for something.

Enough, I'm getting hungry.

Practical Applications of Agile (Web) Testing Tools

This was a practical tutorial by Titus Brown and Grig Georghiu, two accomplished presenters who have given popular presentations at the last two PyCons. Titus opened the presentation, and told us he wanted us to be "test infected" - to make testing a necessary part of our development process. He confessed he feels a sense of physical discomfort when he writes untested code.

He gave us a number of practical guidelines, the most interesting of which were: start small; focus on actual problems; use continuous integration. The goal of testing should be to fail as quickly and as hard as possible: failure is good, because it reveals a need for revision. You should plan for testability, and as your experience level increases you will find that you naturally tend to do this.

Use a hierarchy of tests. The initial tests should run in under 20 seconds; only if those tests pass should you run the full test, which will take longer; then run regression tests, and then (if you have defined them) the acceptance tests. The point is that much of this work isn't done on your time: your continuous integration may be an overnight run, but you don't have to stay while they run. Titus feels that any test that takes over a couple of hours has limited (though definitely not zero) value, due to the lack of immediacy in the feedback it provides.

Taxonomy of testing: unit tests, functional tests, regression tests, UI tests, acceptance tests, integration tests, continuous integration, performance tests. Do you even know what all these are? Which to you need to define and use?

The more complicated it is to run the right tests, the higher the probability they won't get run. Titus talks about different strategies: TDD (Test-Driven Development) and SDT (Stupidity Driven Testing) , which he equated with TED (Test Enhanced Development). The general myth is that you are a bad person if you don't use TDD, but SDT is very important because you can ensure that bugs get properly squashed by adding specific tests after discovering one of the gruesome faults we all put in ur software from time to time.

Tests can be used to constrain your code; they force you to document your expectations, both internally and externally. internally, use more assert statements. Externally, write unit, functional and regression tests as "black box" tests to verify external behavior.

A "test umbrella" removes the need for memory. Everything should report back to a single source. This gives you a way to tell a new developer "run this; if it doesn't work, you have something to fix". You just have to remember to run the tests, not go through some complex sequence of commands that need separate documentation. This automation investment will reap big dividends in both time and quality.

Testing really helps when you want to refactor.

While code coverage may not be a good metric, it's definitely the case that if you don't run every line of code at least once in your tests then the possibility of failure becomes much higher. Line coverage is necessary, but not sufficient. Continuous integration helps you discover less obvious errors, like changes in process you forgot to document.

WSGI intercept is a useful test mechanism because it monkey-patches the Python web libraries.

[Interesting demos; I went for water for the speakers just before they started; my tests failed, and I never discovered whether they were supposed to or whether I had missed some crucial step while I was out].

The scotch tool is a recording proxy that will record everything that passes between the browser and the server: exceptionally useful for capturing AJAX interactions. It can also replay recorded sessions, and ensure that the responses that come back are "the same" (with minor exceptions such as ignoring date changes).

Figleaf is a code coverage recording tool. It saves it results in a pickle-based format, and can perform intelligent comparisons between tests, taking the union and intersection so you can easily discern what still needs testing.

Nose is a system that allows testing with a much lighter footprint than unittest. It will also all other doctests and unittests it can find if properly encouraged to do so, and can be used to add module-level setup and teardown.

Grig then took over and first described and then demonstrated Selenium, a testing tool written in Javascript that runs inside the browser (and probably one of the best-known web testing tools). It's one of the few tools that can adequately test AJAX functionality, and it's the default testing tool at Google). It has several components.

The core is augmented by an RC (remote control) module, and IDE (a Firefox add-on), and a Grid module that seems to relate to parallelized testing.

The biggest issue in defining Selenium tests is locating the required elements in the page. It would be much easier if web authors gave all elements IDs, but in practice you often end up using xpath, which has downsides. Many times there is no other way.

The IDE component allows you to record interactions with web servers, but again you have to be careful to use repeatable event sequences (e.g. pasting is not repeatable); the IDE is probably not as easy to use as Twill, but some AJAX applications are almost pure client-side, with very few server round-trips, and Twill is useless with these as it can't intercept the traffic.

Firebug is an essential component in developing the tests, as is the XPath Checker add-on, which helps greatly in generating the element locators.

The basis of Seleniun tests is a long string of locators and assertions. Selenium RC runs as a reverse HTTP proxy, bypassing the "same origin" cross-site scripting protections that normally insist that scripted content has to come from a single domain. By fooling the browser this way remote control allows you to write test of a remote server inside the local proxy. It also allows you to build simple looping constructs. Of interest to readers will be its ability to export test cases as (rather ugly) Python code. Google has a whole team working on Selenium, and others are also working to improve it, and extend it to broader browser support.

Grig then went on to describe buildbot, a continuous integration testing tool that allows a master system to run build (or indeed any other) commands on remote slaves and aggregate reports from the clients in a simple, readily available web format.

Although some people perform continuous integration on a single platform, buildbot was designed from the start to allow the master to drive the same tests across multiple clients and thereby discover cross-platform bugs that might otherwise go undetected.

Greg ran us through a configuration file. The learning curve can be steep, but once your buildbot is configured it usually needs little maintenance. He also highlighted the fact that someone has to keep on top of buildbots and avoid bitrot by displaying the currrently parlous state of the Python buildbot farm. Many clients are down, and those that are running appear to be failing their tests.

The evening closed with a brief discussion of Fitnesse, which is a higher-level tool that has you define tests declaratively in a language that allows you to refer to the concepts of the application domain; this makes them easier for the customer to comprehend, for example. The tests are maintained in a wiki.

I was beginning to flag a bit by now, so I can't really give you an intelligent summary better than Grig's "it's a Wiki that runs things". Ideally the client will at least understand the test descriptions.

All in all an excellent and enjoyable presentation that not only gave me some great ideas for testing but also left me more determined to introduce it into my web projects.

Getting Started with Pylons/TurboGears 2: Modern Python Web Development

This presentation was as frustrating for me as it probably was for the presenters, Mark Ramm and Ben Bangert. Unfortunately the examples for the first part of the tutorial relied on having up to date copies of lots of packages, and the audience were unable to download them from the Internet as the hotel staff appeared to be having some difficulty connecting 500 geeks to the fat Internet pipe through their wireless infrastructure, and connectivity was extremely intermittent.

Things went from bad to worse when I finally managed to download the required software from the Internet and found that my examples failed: it turned out that only the presenters' machines had the necessary combination of packages. Consequently a lot of the first hour was wasted in trying to organize a thumb-drive bucket-brigade to ensure that all present had the needed stuff loaded. I can understand Mark's anxiousness to present the latest and greatest TurboGears to the audience, but I was there primarily because I wanted to get back into the technology so as to be able to make fair comparisons with Django.

While we were trying to get the load problems sorted out, mark talked about the WSGI interface and its role as a standard in the Python web world. He then talked us through a couple of practical applications, and I ended up with a simple server working based on the reference implementation, wsgiref. We then implemented the same functionality using cherrypy.wsgiserver, observing that the fundamental techniques weren't particularly different. The third exercise didn't actually add much for me, since we just had to get the existing servers to replace the initial output with a listing of the environment dictionary.

Then we moved on to TurboGears. Alas, I ended up typing in code from the handout while Mark was explaining the supporting functionality, and when all was said and done I didn't notice a lot that was new in terms of how you put a TurboGears application together. In one way this is very good, because it means that the TG re-implementation has retained the same API that its developers have become familiar with; in another it's bad, because I didn't feel I actually learned all that much as someone who had already put TG applications together, albeit small ones. Much of what was said was narrative about the state of the software rather than directed towards learning the technology. I didn't have the sample web finished, with nine pages remaining of the handout by the time Mark's part of the session was over.

Ben Bangert then proceeded to explain the differences in philosophy between TurboGears and Pylons. I'm sorry to say that my experience on the first rather-more-than-half of the tutorial left me unable to concentrate sufficiently, and I didn't absorb as much as I would have liked. I don't remember any practical work at all, and I didn't end up with a handout, but I don't know whether I missed it or what.

This was the first of a matched pair of tutorials, the second discussing advanced use of TG and Pylons. I didn't attend it because I was booked on something else in the evening session, so I hope that those who did found that this tutorial had provided them with the necessary background.

PyCon Day 0: A Mixed Bag

Wel, the curse of PyCon struck again this year, and I woke up bright and early to find that my laptop had blue screened. Three hours to the start of my tutorial presentation! Fortunately I was in good shape in terms of backup, having brought my external disk drive with me, and Greg Lindstrom was also able to provide me with a thumb drive containing the PowerPoint slides and PDFs, so I was prepared for not having my own computer available. Fortunately after ninety minutes of repair work the machine was once again in a bootable condition, but it meant that I missed breakfast and was a little later into the tutorial room than I anticipated.

It was difficult for me to judge how well the tutorial went down, so I'll have to rely on people who attended it either commenting here or writing it up in their own blogs. I hope they all remember to fill out the evaluation forms, available electronically as well as on paper. That way I'll have some sort of objective evaluation, but at least a couple of the audience were kind enough to tell me afterwards that they felt better prepared to deal with the conference, which was after all the main idea.

In the afternoon I attended Getting Started with Pylons/TurboGears2 & WSGI: Modern Python Web Development, by Mark Ramm and Ben Bangert, and in the evening Practical Applications of Agile (Web) Testing Tools by Titus Brown and Grig Georghiu. I'll be writing those up in separate posts if you're interested.

In and amongst all that, friends old and new continued to pour in. Too many to mention without fear of upsetting someone, but excellent to see. This conference is going to be a blast, I just know it, and I am very excited about the prospects of the next few days even though it means being away from home. There's a buzz. Or is it just me?

PyCon: Plus ça Change ... *

As is my habit when I am teaching, I arrived in plenty of time (midday today), thinking I would probably be able to help with something or other. Turns out only Drew Moore got here before me (arriving on Tuesday, a luxury I did not have given that I am spending two days at the sprints once PyCon is over).

By 2 pm David Goodger, the chairman, and Doug Napoleone (who has done such sterling work on the PyCon-tech software that pretty much runs the whole deal) had arrived, so we shot the breeze over a few beers and I discovered that the kiloPythonista was a reality—awesome! The organizers (among whom I am no longer counted, except possibly in an emeritus sense) have done a truly magnificent job putting this deal together, and are to be congratulated.

In any gathering of 1,000 people there will be glitches. There are already rumblings about parking (though you can park for $6 a day within a few blocks if you still have the use of your legs). Another issue is that the hotel has apparently realized (after the deal was mentioned on the web) that they might be well advised not to let people to join Priority Club and get complimentary Internet service in their rooms, given that they are trying to share two T-1s among all the hotel guests. Which is pretty ironic, given that PyCon is giving away access to a 45 Mbps pipe downstairs all week. Perhaps we should offer to let them try our bandwidth after we've gone (the minimum term was a month), and see how their customers like free access to a fat pipe. Fat chance, I suspect, since the hotel is a conventional business and not really yet in touch with the long tail.

This conference is going to be huge, in every sense of the word. I have already met several people who are attending their first PyCon, and a large number of old hands who realize that there's a lot of fun to be had just meeting up with people you haven't seen since last year and finding out what they've been up to. As I said to tummy.com's Evelyn Mitchell in the bar last night, coming to PyCon is a bit like coming home.

Some people are wondering whether this increase in size is somehow going to spoil PyCon, and make it less than it has been in the past. I don't believe we need to worry. Despite Guido's rather pedestrian view of the Python community (expressed in an interview I did with him to be published in the conference edition of Python Magazine), I believe it is special. It's special because it collectively works to be different, and welcoming to newcomers (which is essential if the community is to thrive and grow—without that, PyCon would not be the special event that it is), and outgoing, and engaging. As geek communities go, Pythonistas are actually pretty extrovert.

Anyway, it's late and I need to get my beauty sleep, but I just want to record my goals for this year's PyCon. First, I want to forge new relationships on a business and personal level, and reinforce old ones. Secondly I want to ensure that the Python community as a whole knows about Holden Web's Python classes (which are currently being rather ineptly advertised), and understands that we are open for business. Then I want to learn some great new things (including, I hope, how to use Twisted), to teach fifty newcomers the delights of Python, to scope out the PyCon-Tech software with a view to refactoring and migration to the Django trunk, and make a start on a SQL Server back end for Django. Oh, and I am also looking for ideas for more Random Hits articles for Python Magazine too.

Large ambitions, but it's a large conference this year. As Chris McAvoy, whose enthusiasm is largely responsible for bringing PyCon to Chicago, has just written in his blog, hooray for PyCon!

* "plus ça change, plus c'est la même chose" — "the more it changes, the more it's the same thing", usually translated as "the more things change, the more they stay the same" (Jean-Baptiste Aphonse Karr, Les Guêpes, January 1849)

One kiloPythonista?

With only days to go to the end of pre-conference registration the numbers for PyCon 2008 are tantalizingly close to the thousand mark (as I write, 941 registered, and the possibility of a few additions in the pending changes). It would be nice to be present at the first kiloPythonista gathering, and there is often a minor surge at the very end of the registration period. Whatever happens, this conference represents a new high-water mark in Python interest.

Since I missed out on the (apparently very entertaining) Twisted BoF last year I have attempted to make amends by setting up a session in which I try to learn Twisted in public. I'm not sure how much I will manage to learn, but I hope it will be entertaining. We might even manage to generate some useful notes for others faced with the same task.

There's been a lot of talk about the catering on the organizers' list recently, leading to no real conclusion except that you can't please everyone. I am sure nobody will starve, though, and that's the main thing.

The Chicago Python group are meeting on the Thursday evening before the conference proper starts. I wanted to join them, but I am signed up for an evening tutorial which continues until after the end of the meeting. I hope they will publicize the location of the after-meeting meeting, and maybe we can share a drink.

Plans to promote a bid for PyCon to return to DC in 2010 have been shelved, as I don't have the time to do much about them. I was skeptical of the value of using a conference organizing company, but several of those who have been involved in the planning say that the assistance has been invaluable. It looks as though there may be a Python BarCamp in DC in the fall, though, and I am now setting my sites a little further out for the next DC PyCon.

All in all I am really looking forward to this PyCon, number six in the series. Hope to see you there - please look out for me (my photo is in my Blogger profile, among other places) and introduce yourself!

PyPy Catching Up ...

An interesting post on the PyPy (Python written in Python) blog explains that PyPy has finally, on carefully selected tests, benchmarked ever so slightly faster than CPython (1% faster on one test, 11% faster on the other).

It's also interesting to note that a 20% speedup is achieved by method caching, which avoids the need to create a new bound method every time a method is referenced relative to an instance. Apparently the same technique would be usable in CPython were it not for certain C modules directly modifying the __dict__ of new-style classes. I wonder if CPython could obtain an equivalent speed-up, and if so whether it would be worthwhile making the guilty C modules restrict themselves like PyPy does.

I have high hopes that within a year we will regularly be seeing benchmarks that show PyPy outperforming CPython.

If you can't wait for that, and can live with heavy restrictions on your use of the language, try RPython (another product of the fertile PyPy project, and one in which Stackless Python's Christian Tismer had a hand) which on occasion benchmarks faster than C!

Industry Support for Python Broadens

In a move of significance to all Python users Sun Microsystems has broadened its support for dynamic languages still further. There was some interest in their announcement less than six months ago that they had hired Charles Nutter and Thomas Enebo, developers of JRuby.

Today we learn that they have recruited Frank Wierzbicki and Ted Leung. Frank has been one of the two principal maintainers of the Jython project for a good long time now, and has brought the language forward to the point where it supports the newnew-style classes, a significant milestone in compatibility with CPython. Ted is a long-time member of the Apache Software Foundation who was a member of the OSAF's Chandler team for the last four years.

Sun was a great employer for me (I was badge number 1536, I believe, starting back in 1985), and I hope that Frank and Ted enjoy working there. It's certainly a great place to meet people with exceptional intelligence, and the fact that both Sun and Microsoft are now visibly taking a serious approach to dynamic languages in general and Python in particular is terrific.

Sun and Microsoft are both sponsoring this year's PyCon, for which registrations recently topped 900. If you want to meet them there then you still have four days to register at the pre-conference prices!

Back to the Drawing Board for OOXML?

Even some pretty shady manipulation of the various national committees doesn't appear to have helped Microsoft much in their bid to get OOXML, the proprietary format describing Office documents, declared an international standard. The ISO/EIC Ballot Resolution Meeting in Geneva has ended with what can at best be considered an inconclusive result, with only eight of the 'P' countries and ten countries altogether registering any kind of vote on the main matter put to them.

The real problem here is Microsoft's attempt, with the assistance of ECMA, to put the standard through the fast track process; that was clearly not appropriate for a standard of such complexity and political significance. There were over 1,100 dispositions to be decided at the meeting, and only 20 substantive dispositions received any measure of individual attention.

The meeting's convenor attempted to reach a conclusion by bundling together 900 dispositions that had not been discussed and asking the delegations to vote on each without discussion. Unfortunately this led to over 80% of the delegations either refusing to register a vote or abstaining on all dispositions.

The real problem is that rather than being a standard driven by the requirement to describe structured documents of all types, OOXML is simply an XML-based description of an existing proprietary format.If Microsoft truly want to embrace standardization they should now get behind the Open Office ODF format and support it with all the weight they can. This seems unlikely given their prior position on ODF support, which can at best be described as half-hearted. They apparently have a lot of work to do to make OOXML acceptable.

Technically there is now a 30-day period in which the ISO/IEC members can vote to approve the adoption of OOXML. I doubt even Microsoft has sufficient clout to persuade them that this travesty or a process represents a satisfactory basis on which to build an international standard.

False Economy?

I went to a meeting the other day with one of my clients and two third-party developers. We were meeting at the developers' offices, in a town I'm not familiar with, so when I arrived I looked for the closest parking garage and parked there.

Arriving at the offices, the opening smalltalk included parking. Both my client's representative and the two developers had parked in two-hour free parking areas, and when I said I'd used the parking garage eyebrows were raised and one person asked me "how much does that cost?". I replied "I don't know, I didn't even think to look".

The meeting lasted four hours, so half way through the other three participants in the meeting had to take ten minutes or so out to juggle their cars and avoid tickets. On my return to the parking garage I discovered that the charge equated to approximately three minutes' billable income, which I was happy to write off. Assuming the other three participants are similarly expensive their decision may have saved them personally a dollar or two, but it probably cost the client over a hundred bucks in lost time.

Yet I'm the one who's regarded as stupid for paying to park so I can happily take as long as I need to meet without interruption. I'd like to think my clients pay me the rates they do so, among other things, they don't have to pay me to go out and move my car in the middle of meetings. A couple of dollars out of my pocket is well worth it if the client gets (and perceives) better value for money as a result.

EU Slam Microsoft with Further Fine

The saga continues. The European Commission have fought Microsoft to ensure that, in Europe at least, they have to play by the rules rather than making them up as they go along. Unlike the US government (which not only refused to declare them a monopoly despite their domination of the desktop but also failed to properly enforce the rather tame sanctions they imposed), the EU has repeatedly told Microsoft to stop behaving in an anti-competitive way.

The latest action, in which the Commission sued Microsoft for failing to comply with previous antitrust decisions, has resulted in a €899 m ($1.35bn) fine. Though it has only led to a 12c drop in the stock this represents about a tenth of Microsoft's annual profits, and may finally give the company reason to think seriously about complying with court rulings rather than ignoring them.

This is the first time in 50 years that such action has been taken by the EU, because every other company losing such an action chose to comply with the spirit and the letter of the decisions.

Microsoft faces further trouble with the EU, which recently started a new investigation into Microsoft business practices relating to their dominance in the word processing and spreadsheet arenas, and to determine whether the company unfairly tied Internet Explorer to Windows. The recent announcement about API sharing will also come under scutiny, and the European Competition Commissioner, Neelie Kroes, observed it's the fifth time that Microsoft has made an announcement about improving interoperability. She said "Talk is cheap. Let's wait and let's find the reality in this case. They have to deliver and implement."

PyCon is International

Various national conferences have started to use the PyCon name, and this has now been formalized by making the various national sites available under the pycon.org domain. So now you can go to:

• br.pycon.org
• eu.pycon.org
• fr.pycon.org
• it.pycon.org
• uk.pycon.org

That's a long way for a community conference to spread in five years! The eu conference is a redirection to EuroPython, which is a trans-national conference and (if I am not mistaken) was actually started the year before PyCon. But it's nice to have them in the family.

Thanks to Marc-Andre Lemburg for maintaining the domain. Let's hope to see more PyCons as the Python's influence spreads further and its popularity continues to increase.

Don't Compete and Microsoft Won't Sue

It's obvious that not everyone is happy about Microsoft's most recent attempt to woo the open source community by releasing API details and offering them for use with a covenant "not to sue open source developers for development or non-commercial distribution of implementations of these protocols".

It isn't yet clear exactly what qualifies as "non-commercial distribution", but it's an interesting approach: Microsoft has always complained (sometimes unjustifiably) about the GPL. Now authors of open source products can use the newly-released information as long as they don't gain commercial benefit by doing so. This could be seen as opening up, or simply as a nice piece of spite. It will definitely result in the development of interoperable projects that will take market share away from Microsoft, leaving software dollars looking for other places to be spent.

I suspect that a lot of people will say that this doesn't go far enough, but it's a sea change in Microsoft's approach and has doubtless taken lots of internal discussion. If you develop a commercial product based on Microsoft's patented technologies then you will still need a license from them before you go to market. As long as you believe software patents are reasonable this is nothing to complain about.

Personally I have never thought software patents reasonable, but Microsoft are to be congratulated for trying to promote their technologies in a more open way. Let's hope this is just the first step to fully open standards, only twenty years too late.

Phatch, the Photo Batch Processor

Phatch, written by Stani (author of SPE, Stani's Python Editor) is probably the nicest open source program I've used this year. I'm no great shakes with graphics, but I wouldn't even have been able to think about doing the transormations to my profile image that you see on the right.

The graphical interface makes it easy to perform the same operations on a bunch of different images in the same directory, and the flash image that comes up at the start always gives me a smile. Still in need of development, but definitely recommended, particularly if you have a lot of images to work on.

Needless to say, this little whizz is written in Python.

Biggest PyCon Ever Starts March 14

It now appears definite that this year's PyCon is going to be a monster. As of nine o'clock this morning there were 755 registrations, after the usual spike at the end of the Early Bird period.

Murphy's law dictated that registrants started experiencing problems with credit-card payments in the last 36 hours of Early Bird due to problems with the third-party processing service. Right now there are over 100 pending credit-card registrations! If you experienced a problem and did not manage to complete your registration, fear not—you will be charged the Early Bird price if your registration was started in the right time frame but you could not complete it due to site problems.

I'm buzzed about this conference. The tutorials are getting packed—several, including my own, are nearing the 50 mark—and there are still over three weeks to go. I believe this is going to be the largest-ever gathering of Pythonistas in the world. Be there, or be a four-sided regular geometrical shape.

I'm thinking about trying to do some audio interviews while I'm there. Who would you most like to hear from? You can see the sign-up list (excepting those who requested no publicity) here.