PyCon 2009 Presentations

I've just been thinking about planning for PyCon, and I have a lot to do. The tutorial schedule is a little bare right now, but I am hoping it will be fleshed out soon. There are over 30 tutorials, two of which I am involved in.

Firstly I have to write Python 401: Some Advanced Topics (not my own choice of title), which is supposed to discuss iteration, generators, descriptors and metaclasses. These are all features of Python that could stand some more explanation, but I'd like to put a twist on the material and have the audience suggest potential applications that we can discuss as a part of the tutorial.

There's a second tutorial, Python for Teachers that I'll be presenting jointly with Kirby Urner. The idea behind this one is to try and enthuse those involved with formal education about Python's potential for teaching mathematics of various kinds. Kirby has a lot of experience doing that, and I am supposed to provide irreverent questioning and keep Kirby headed in the right direction. He's the education side, I represent the geeks, if you like. I'm really looking forward to it, but I owe Kirby email about those plans.

Then I have to give an invited talk which I believe will be a plenary session, though that hasn't been confirmed yet. My working title for this is Leading the Python Tribe, in which I want to investigate the shared sense of direction in the Python community, in so far as it currently exists, and offer some thoughts about the roles of the Python Software Foundation.

I am also hoping to run an Open Space session along the same lines as last year's Teach Me Twisted, but this time looking at web testing - an area I would really like to learn more about. I think I have Grig Gheorghiu interested in helping out, but he's a busy guy and hasn't yet committed. If anyone can rope in the knowledgeable web testers we need it's Grig.

To top this all off there'll be the PSF members meeting to chair, sundry sessions in the bar, meals with friends old and new, and all the other wonderful things that happen at PyCon with absolutely no formal scheduling. So I expect to be busy, but I also expect to enjoy myself hugely. See you in Chicago?

Sun Developer Network Podcast

It's taken quite a long time to get through the production cycle, but here's a podcast of Ted Leung and me discussing Python 3.0 migration. Something to listen to on the commute?

We're TravelNow.com, and We Fail

Don't you just hate it when an advertised email address fails? Particularly so when you've sent the email to notify a company about a problem with their web site? It's surprising how some quite large companies fail to pay proper attention to the basics, and I came across a perfect example today.

I've just booked a trip with Jet Blue, and wanted to add a car rental reservation to the booking. Let's ignore for the moment the tedium of having to enter the same details twice due to a lack of integration between Jet Blue and TravelNow, as the price of lower-cost flying. Alas, when I click the "printable version" link after completing the rental reservation I see a pop-up window saying "Unrecoverable error: if this persists please email webmaster@travelnow.com". Being a good web citizen and having a certain fellow feeling for anyone who struggles to keep the Intarweb working I dash off an email with the URL and my reservation details, only to receive the following missive:

Oh well, these things happen in the best-regulated domains, I guess. An email to postmaster should sort it out. I mean after all, RFC2821 explicitly says

SMTP systems are expected to make every reasonable effort to accept mail directed to Postmaster from any other system on the Internet. In extreme cases --such as to contain a denial of service attack or other breach of security-- an SMTP server may block mail directed to Postmaster. However, such arrangements SHOULD be narrowly tailored so as to avoid blocking messages which are not part of such attacks.

so that's bound to work, right? Wrong:

So I guess the lesson here is "we don't care if there's anything wrong with our web site, and we haven't bothered testing our customer feedback channels". If this company has stockholders, they should be asking questions.

Yes, I have verified my own domain's webmaster and postmaster addresses. I thought it would be a good idea before I made this post. Can you say the same?

Edit: A colleague pointed out that I had amused her by including a typo in the email address, which I have now fixed. So I am not as good at checking the basics as I'd like to be myself ...

A Pythonista Lending Group?

I've finally taken the plunge and become a lender with Kiva. It got me wondering whether there might be some potential for a lenders' group in the Python world. I know from personal experience that it contains many people with philanthropic and charitable instincts. If you'd like to join in, let me know.

Python 3.0 Is Out

Even though I am on vacation this is worth a quick note. After long efforts by many developers Python 3.0 was released today!

I posted a short article a while ago about 3.0 (in)compatibility, but the differences between 2.6 and 3.0 aren't so great. It's perfectly possible to write 3.0 code that will run on 2.6 too, as most of the language hasn't changed at all.

The preferred strategy for writing code that runs on both versions is to write in 2.6 and then apply the 2to3 converter and verify that it produces a correct 3.0 program. There's no guarantee that it will, so you may need to paraphrase the 2.6 code a few times before you get a transatable program.

Once all the third-party modules you and extensions you rely on are 3.0-ready, and you no longer have clients requiring 2.6 version of your software, you can simply drop the 2.6 compatibility requirement and start to make use of the few 3.0-only constructs that have been introduced.

Religion is not Python

About six months ago I made a post called Python is not a Religion, discussing some aspects Python advocacy in response to some comments by Doug Napoleone. About two weeks ago I see a new comment (by Jonathan) appear against that piece:

I noticed that your Blogger profile says "Catholic tastes" for your favorite music. Are you a Catholic, and would you be interested in joining the recently formed Association of Catholic Computer Programmers?

Firstly, I should like to point out that I am not a member of any church. I have no interest at all in worshipping anybody else's imaginary friend, though I try (with varying degrees of success) not to make too much fun of other people's delusional belief systems. I have my own to maintain. In this case I am going to make an exception. Without my really knowing why, this particular comment got right up my nose. I suppose, now I think about it, it's the shallowness of understanding that led the person to post the comment. Google has a lot to answer for.

Secondly, the word "catholic" in my profile is specifically used in the sense of "free from provincial prejudices or attachments", but apparently some Catholics don't appreciate that the name of their church has a broader (and far less exclusionary) meaning, dating back to the days when you were Catholic or you were on the rack, buddy. The Catholic church is so named because it was originally the "general or universal" Christian sect. Of course Henry the Eighth laid waste to that when he started the Church of England, and as far as I can see the Christian faith has more branches than the average oak tree. Whatever.

The Association of Catholic Computer Programmers (Catholics interested in software engineering best practices) appears to be a little schizoid. Are they a social network, a religious institution, or a movement? Some of the discussions are interesting, with comments like "I think we should start a project to benefit a charity of our choosing and put our collective talents as Catholic SEs to good use", which makes me wonder what skills Catholic software engineers have that I don't.

The same guy who started ACCP has started "a pro-life network", though apparently it have only four nodes so far. There's a pretty badge people can use "to identify themselves as being pro-life". Well, of course I am pro-life. Without life I'd be nowhere, and you wouldn't be reading this drivel now. In fact I find it difficult to understand how any living being can be anti-life, with the exception of the dysfunctional. But of course there's a hidden semantic trap there: the opposite of pro-life isn't really anti-life at all, it's "murderer".

The most revelaing thing of all to me is that neither the ACCP nor the above-mentioned network are prepared to do what enabled the commenter to make his or her comment in the first place. They require sign-up to make any sort of input to their site, while my blog allows anonymous comments from anyone (though as a spam reduction measure they aren't immediately posted). So the Catholic Programmers aren't at all catholic in their approach to discussion.

I know a lot of good Christians, but I don't want to join that club. Sorry. I suppose Jonathan can only hope there's no such thing as bad publicity.

LodgeNet Censoring python.org?

So, here I am at this hotel, using their network to browse the web as one does, and I suddenly see this web graphic notifying me that my network access has been blocked for ten minutes because "[the] server has detected questionable traffic originating from this computer".

At first I was concerned that perhaps I had unwittingly picked up some virus or other malware, so I made a call to the "support" team to see if they could tell me what was wrong with the traffic. Of course, I might as well not have bothered, since they couldn't tell me anything about why this might have happened without me being "naughty" in some undefined way. After ten minutes the service came back on again, only to be lost once more as soon as I retried a web access. So it seemed to me there was something extremely significant about the URL I was trying to access, which was actually the documentation for the PIL ImageDraw module.

A further support call elicited the information that the help desk couldn't do a bloody thing about anything, had no idea why the content was being blocked, and weren't interested in helping to change whatever bizarre piece of policy was responsible. Several times they mentioned that "the server will cut you off if you are downloading, there's nothing we can do about that", leaving me somewhat perplexed about what they thought the Internet was for, but of course I assured them that I had only been "browsing," which is apparently different.

It would have remained a curiosity had I not then received the same response when I tried to visit the Python programming language home page. Now suddenly the position is different, as I am the chairman and an officer of an organization whose web site is being subjected to censorship. So I have sent the customer service department at LodgeNet the following snotty letter. We'll see whether I get any response, but I am not holding my breath.

Sirs:

As a resident at the Town and Country Resort Hotel on San Diego, California, I have been a consumer of your company's service. Several times I have had my service blocked, seeing the attached graphic.

Calls to your customer service department elicited no useful information as to how I could stop this unwarranted blocking of my service, and the staff involved, while sympathetic, were completely unable to help (and indeed appeared to know nothing at all about the policy responsible for this). The only suggestion they could make was that "downloading" would cause the service to be blocked: hardly helpful, since I was merely browsing the web and handling email.

My fourth call to your support desk was put on hold and then unilaterally terminated. This is completely unacceptable.

Experimentation demonstrated that your service blocker appeared to come into play when the certain URLs (among others: my patience was limited) were accessed. I would emphasize that my use of bandwidth was *not* excessive; the previous evening I had transferred a couple of PowerPoint and PDF files, but no malicious content of any kind was involved.

http://www.pythonware.com/library/pil/handbook/imagedraw.htm (documentation for a graphics package)

http://python.org/ (the home page for the Python programming language).

This last site is of particular concern to me as I am the Chairman and a Director of the Python Software Foundation, the organization responsible for the site.

I would like to know why you are censoring our web site, and what you propose to do to make sure that its content is available to all who wish to access it.

Sincerely
Steve Holden
Chairman, Python Software Foundation

Don't Use Python 3.0? Really?

It's difficult to say "don't use Python 3.0" without being rude to the developers, as the code seems to be of very high quality. Now that Python 3.0 is almost here (release candidate 2 is a week old as I write) we should encourage people to use it, but only under well-defined circumstances. Here is some text I put together for inclusion in the release notes, to give people guidance about when and why it is and isn't appropriate to use 3.0. I would welcome further ideas to refine this text.

Readers should note that this release has significant backwards incompatibilities with the 2.X series, both at the language level and also in the C APIs available to extension module authors. These have been introduced by design, with the goal of improving the language in the long term, but this situation has implications for production use.

Furthermore it is likely that there will be a considerable delay before some authors of Python packages and extension modules start to provide 3.0-compatible releases. There are significant hurdles to be overcome in migrating extension modules from Python 2 to Python 3.

For these reasons you are NOT recommended to rely on Python 3.0 as your main production Python implementation until you know that all supporting code is available in compatible forms. While every care has been taken in the production of the Python 3.0 release, and the release team feels that the system is of a quality comparable with the recently-released Python 2.6, the latter system is currently the recommended version for production use.

I hope that clarifies things, but I would welcome further ideas to refine this text and make it more helpful. One of the primary reasons for using 3.0 is to get a migration strategy ready. But 3.0 is undoubtedly not for everyone, at least not just yet.

Phrase From Nearest Book

Well, here's a turn-up. I spotted this meme on Grig Georghiu's blog.

• Grab the nearest book.
• Open it to page 56.
• Find the fifth sentence.
• Post the text of the sentence in your journal along with these instructions.
• Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

As luck would have it I am on a trip to San Diego and the only book I have with me is Rapture for the Geeks by Richard Dooling. [Disclaimer: Richard was kind enough to send me this book in return for a programming favor I did him some time ago]. By the most amazing coincidence, when I examine the bookmark I placed to make a telephone call I see I am currently reading page 56! Here's the fifth sentence:

If he is gay, man needs man; otherwise, man needs woman.

This doesn't actually represent the compelling nature of Dooling's prose particularly well. For that I like the next two sentences rather better:

Not photos or Web clips or blog entries or text messages or e-mails or even video clips taken of or sent by woman. Man needs woman in the flesh, in meatspace, where he-meat meets she-meat to beget meatlets.

Anyway, it will be interesting to see what others come up with.

United.com Broken

So I have this trip coming up to San Diego, and I fly tomorrow. I get this friendly little Web-2.0 e-mail from United. It comes from an address in a subdomain of p0.com, telling me that United are using Yesmail, and it includes a link for me to follow to "view my complete itinerary". They even have it link to a server in the p0.com domain so my browser doesn't think this is a phishing attack. The only problem comes when I click on the link, and repeatedly see 

Connection Interrupted

The connection to the server was reset while the page was loading.

The network link was interrupted while negotiating a connection. Please try again.

"Not to worry", I think to myself, "I'm a United frequent flyer (God help me), so I can just log in to my Mileage Plus account and check my itinerary there". Of course this wouldn't have been much of a post if that had worked, but good old United didn't let me down. Every attempt to log in gives me

Redirect Loop

Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

The browser has stopped trying to retrieve the requested item. The site is redirecting the request in a way that will never complete.

Attempts to retry just bring up the same message again. Wonderful. Let's hope the people who run the web site have nothing to do with keeping the 'planes flying.

[PS: Fortunately this appeared to be a transient condition. The site now responds normally.]

Final "Introduction to Python" of 2008

As the year draws to a close Holden Web is pleased to remind readers that its final public "Introduction to Python" class of this year will be held from December 9-11 at our education center close to Washington, DC.

There are several hotels conveniently located within walking distance, and we provide breakfast snacks and lunch each day.

Purchase places on-line by credit card or using PayPal.

Thanks to all students who have already attended this class. We look forward to meeting you again as our range of course offerings increases.

For Would-Be Python Users

If you are new to Python, or know someone else who is, you (or they) could do a lot worse that visit Richard Dooling's Seven Minutes to Hello World web page. It explains about the easiest way to get Python up and running on your Windows XP computer.

I am pretty sure it will run for Vista too, the only reservation I have being the need to run the installer as an administrator if you want to install Python for all users. Since most Vista systems are single-user machines anyway, this isn't likely to be a big issue.

World Plone Day? Communicate, Dammit!

So just now, somewhat after midday Eastern Time, I discover that today is World Plone Day.

Forgive me for saying so, but as Python Software Foundation chairman I should have expected to hear about this event a little in advance. I would even have welcomed the opportunity to lend PSF resources towards publicizing it, seeing as how Plone is one of Python's showcase applications. Alas apparently nobody thought to try and use the PSF as a channel. So I hope World Plone Day goes well this year, and that it doesn't pass everybody else by as it did me.

For the record, the PSF runs its own blog, whose articles are well-syndicated on the web, and we are also usually happy to put Python-related events as front-page news on the python.org web site.

If you are running some newsworthy event, mail the webmasters (if you aren't savvy enough to work out the correct email address, webmaster at python dot org should get you there) to avail yourselves of the opportunity! However can we achieve world domination if we continually hide Python's light under a bushel?

A Joke, or an Accident?

I can't decide whether Greg Wilson posted this as a joke. In case he didn't, and whatever error induced this behavior is fixed by the time you click on the link, I attach a current screenshot. Click on the image to look at the title and see what amused me.

Either way I think it's funny. No, it doesn't take a lot to amuse me. I guess I'm just a geek at heart.

Stani Wins Coin Design Competition

If you subscribe to the Planet Python feed you will already know that Stani (this is to the best of my knowledge the only name he advertises), the author of the well-known Stani's Python Editor, just won a competition to design a new Dutch five-Euro coin.

Congratulations, Stani, and not just for an exceptionally creative use of open source software (including Python, naturally), but also for a fine demonstration of its ideals. What a pity you can't release the coin under a copyleft agreement!

It seems to me this is just another area where the demonstrable abilities of open source are only just beginning to make an impression. You ain't seen nothing yet!

Norton Antivirus 2009 on Vista is a Liabilty

Today, for the second time in less than two weeks, the first attempt to collect mail after a reboot cause the appearance of a rather unpleasant notification dialog indicating that my Antivirus software has stopped working:

The first time this issue occurred I spent almost four hours online with the Symantec support technicians, having my software removed and re-installed several times without any resolution of the issue. At that point I had to go out, and when I returned the issue was somehow gone. I assumed that some component had been automatically updated, and continued on my merry way.

Today, we went through the same remove/re-install rigmarole with no more success. After the technician (who was controlling my computer remotely) went quiet for three or four minutes, then he closed the browser windows and disconnected the support session. Thanks very much.

The really annoying thing is that this is a known issue that Symantec claim will "be fixed in the next support upgrade", but there is no information about when this upgrade will happen, and neither does there appear to be a workaround I can apply until the fix appears, short of falling back to Norton Antivirus 2008. So I suppose that's what I'll have to do.

I'm guessing there are a lot of other people experiencing the same issue, because I started out as eighteenth in line for support. I am now next, and looking forward to going back to the prior release. But that was installed by a Symantec technician in response to a long-forgotten problem with the 2008 product, so I may not be out of the woods even then. If Symantec think I'm renewing at the end of my current subscription period they have another think coming. I have already spent almost a full day without email twice because of this issue, and I've had enough.

For some reason a number of posts were stuck in the blog as drafts. This one is from October last year.

It's Here!

I may be an old fart, but I know when a 'phone is going to be cool. To think I was actually the guy who registered "gPhone.com" (for a client). It would be nice to see Steve McCarthy and Guy Jazynka get something for the name. They might even buy me a pint!

At least I get a chance to play with Chrome. So, who knows where the Python interpreter is?

Shame I am feeling so ropey, I'd have loved to spend the evening playing on the Intarweb. Maybe tomorrow I'll blog from the 'phone.

Django Tip: Non-editable Fields

Yesterday I explained how to subclass a DateTimeField, adding a pre_save() method to make it suitable for use as a row creation or modification timestamp. If you do this, however, you will observe that unless you add an editable=False argument to the field instantiation call they will appear as editable fields in any ModelForms containing them. It would clearly be more desirable to have these fields non-editable by default, which can easily be arranged by extending their __init__() method and changing the default value of the editable argument.

The code below also renames the fields more suitably. You can use a similar trick to create other non-editable field types, and also to change other instantiation defaults.

class ModificationDateTimeField(models.DateTimeField):
  def __init__(self, editable=False, *args, **kw):
      models.DateTimeField.__init__(self, editable=editable, *args, **kw)
  def pre_save(self, instance, add):
      val = datetime.datetime.now()
      setattr(instance, self.attname, val)
      return val

class CreationDateTimeField(ModificationDateTimeField):
  def __init__(self, editable=False, *args, **kw):
      models.DateTimeField.__init__(self, editable=editable, *args, **kw)
  def pre_save(self, instance, add):
      if not add:
          return getattr(instance, self.attname)
      return ModificationDateTimeField.pre_save(self, instance, add)

Django Tip: Field.pre_save()

Here's a little thing that can help you to improve the flexibility of your models.

Rather than trying to extend or override Model.save(), if the special action you require only affects one field then define a subclass of the required Field type and define a pre_save() method on the subclass. This will override the default method, which simply returns the attribute value.

I spotted this by following up a discussion of the planned removal of the DateField's auto_now and auto_now_add attributes, which you can assert to set a modification or creation date respectively on rows. Some developers suggested putting the functionality in a decorator that could be used on the model's save() method, but Jacob Kaplan-Moss preferred a trivial subclass:

class AutoDateTimeField(models.DateTimeField):
def pre_save(self, model_instance, add):
    return datetime.datetime.now()

Unfortunately this is just a little bit too trivial, since pre_save() is required to set the attribute value - every time for auto_now and on first save for auto_now_add. Fortunately the extra code required is pretty small. pre_save() receives three arguments: the first is the field instance, the second is the model instance and the third is a flag which is True only when the save() call is adding a new row to the associated table.

class AutoNowDateTimeField(models.DateTimeField):
  def pre_save(self, instance, add):
      val = datetime.datetime.now()
      setattr(instance, self.attname, val)
      return val

class AutoNowAddDateTimeField(AutoNowDateTimeField):
  def pre_save(self, instance, add):
      if not add:
          return getattr(instance, self.attname)
      return AutoNowDateTimeField.pre_save(self, instance, add)

Technically it would be cleaner to use super() to access the superclass methods from AutoNowAddDateTimeField, but as a proof of concept this works fine.

As it happens the auto_now and auto_now_add attributes of DateTimeField were not removed before the release of Django 1.0, so we are likely to be stuck with them until 2.0. But the same technique is useful for any field that needs to receive an automatic value when a model instance is saved.

Python 2.6 Released

Thanks to release manager Barry Warsaw and a cast of thousands! Get it from the usual place.

Scratching Setuptools Itches

Interestingly, two different people have announced setuptools replacement/enhancement products on the same day. Maybe we will eventually see some such functionality in the Python standard library. It's obvious there's a lot of pull for the feature set, even though different people have different complaints about the implementation.

The Other Side of the Bike Shed

I've long used the C. Northcote Parkinson bicycle shed discussion example to try to focus on the big issues, and suspect I may have been the one to introduce it to comp.lang.python (though someone with superior google-fu would have to confirm or deny that assertion).

It's always a temptation to expend most energy on the issues we feel comfortable with, and rush the discussions on the areas outside our area of competence even though they are the ones we should rationally spend time discussing, grappling with issues and trying to understand.

The current Bush administration proposal, on which they demand action from Congress within a week, sees the other side of the bicycle shed. Urgent action is alleged to be needed, and it seems that many in government are prepared to rush a decision through because they don't really understand the issues.

The fact remains that it's an unconscionable bid to bail out the fat cats whose corporate greed, with the connivance of the SEC, sent the economy down the shitter in the first place. Personally I'd rather see the discomfort spread about rather more democratically, with those who have been making money from their investments in ill-advised mortgages now taking losses still not comparable with those of the often innocent borrowers who are losing their homes by the thousand.

Remember, the value of your investments can go down as well as up.

PyCon Tutorials, 2009

So, the call for tutorials has gone out, and I am thinking (yet again) of offering at least one. In the past my tutorials have been of an introductory nature: the DB API, wxPython, and last year I taught Python itself (or enough of it for newcomers to be able to understand conference talks) in three hours.

So, what would people like to see this year? I'm minded to do something a little more advanced this time, so I can feed it into Holden Web's public course schedule.

The Photo Meme

Instructions: Take a picture of yourself right now. Don't change your clothes, don't fix your hair - just take a picture. Post that picture with NO editing. Post these instructions with the picture.

From a post by Jeremy Hylton.

Python 2.6 First Release Candidate Due Today

To maintain the fairly aggressive release schedule Barry Warsaw and the Python developers have decided to decouple the 2.6 and 3.0 releases, so 3.0 will now be slightly delayed - maybe by a couple of weeks. This gives them the freedom to bring out 2.6rc1 today, with the intention of brings 2.6rc2 out on Wednesday September 17 as originally planned, and hitting the October 1 final release date.

Since 3.0 isn't intended for production work I think this decision is the right one. There are still a few release-blocking issues on the 3.0 code, and there's little point trying to release it before quality is acceptable.

It will be great to see the next new release of Python. If you have any chance at all to test the release candidates you too can help to improve the quality of the eventual release.

PyCon Is Coming ...

Incredibly, PyCon 2009 is already looming. The call for papers should be opening soon, and the conference needs volunteers like never before. I've just finished an article on that topic for Python Magazine, but that won't be published until November, so keep your eyes on the PyCon blog for more news.

Don't You Hate It ...

... when you [use a well-known search engine] for advice about a problem and the first hit is something you wrote several years ago? Either you should have remembered you've had that problem ever since you first wrote about it, or you should have remembered how to solve it.

Holden Web on Django Trunk Again

The main holdenweb.com site is fairly simple, so I figured it would be a short job to migrate it to the latest version of the Django trunk (r8301 when I started). While I felt it was unlikely I'd unearth any errors it's the least I can do to help the developers as they charge towards the first release candidate for 1.0.

It turned out to be relatively simple, but with an unanticipated glitch. Having made what I thought were all necessary changes, most of which simply involved altering

from django import newforms as forms

to

from django import forms

I tried accessing the site and got a lengthy traceback terminating in

ImproperlyConfigured: Error while importing URLconf 'urls': cannot import name newforms

This would have been fine, except that the traceback didn't actually give me any information about the module that was causing the problem. Ultimately I traced it to the registration add-on, which needed the same change making to it as did my own code.

I'm pretty impressed, even on a small site, that such large changes to the code needed such small changes to my application before it worked. Kudos for code quality to the Django developers. But it would be nice if the error message could be more informative.

One-Day Python Classes

Holden Web's third Introduction to Python class starts on September 9. It runs from Tuesday through Thursday, and I could easily use the training room for an extra day if I had a suitable topic for a one-day class. The question is, what would people want to learn? More specifically, what would they pay to come and learn?

I have a couple of ideas for popular topics, but your suggestions are also welcome. And since there's only over a month to go, better hurry! Thanks.

Microsoft at OSCON

The last session I attended before leaving for the airport was the Friday morning plenary session. Sam Ranji of Microsoft announced several new initiatives (documented in his Port 25 blog), including sponsorship of the Apache Software Foundation. This doesn't mean Microsoft are abandoning IIS development.

After the keynotes were over there was a Q&A session, but it seemed like the AV technicians (who seemed to do a pretty awesome job overall) hadn't set up monitor speakers, and so those on stage found it difficult to hear the questions. Which were mostly for Sam, and sometimes a little hostile!

I tried to ask him how it would be possible to change the corporate ethos so that stupidities like the recent OOXML debacle aren't repeated, but I'm not sure the desire to help survived the mangling by the sound system. Microsoft have been a sponsor member of the Python Software Foundation for some time now, and they are as welcome as all our other members.

It was clear, listening to Sam speak with delegates after his talk, that he "gets" open source. It's frustrating to read people complaining about Microsoft without being prepared to engage and argue that open source and full interoperability can actually help the company meet its business goals more effectively. By doing so we can help Sam get the ears of the business-line managers, and thereby promote open source from within.

I left him with a business card and an offer of assistance. It remains to be seen whether anything comes of it.

Where's Sean Reifschneider When You Need Him?

Another conference, another sucky network. OSCON's wireless network hasn't performed as well as I'd have expected it to from my experience four years ago. With roughly 2,500 delegates in a large area there's no reason why I should be unable to associate with an access point.

Sean's setup and management of the 2007 PyCon in Dallas remains the best conference network I have used.

All this seems very effete, of course, given that ten years ago there was no such thing as a public wireless network. But as technology moves on expectations move up, and many people this week have suggested OSCON's network will need to improve for next year.

Is My Nose Borked ...

... or do all Portland Conference Center hotels smell of stale fried food?

I am having a great time, and the PDXers are wonderfully hospitable, but I suspect the hotel staff need to take a week off and smell the places they work with a fresh attitude.

I have stayed in many hotels that don't have this problem, but the two I have visited so far this week (including the "conference hotel" I am staying in) aren't the freshest environments I have ever encountered.

Test the Betas, Already!

OK, if you saw the last mention of the Beta 2 release of Python 2.6 and 3.0 you might have got this message, but the subject line makes it pretty explicit.

Every time a new release comes out it goes through the alpha, beta and release candidate stages without enough people actually downloading it and making sure their applications run. (Then, surprisingly often, within a month or so of the final release someone comes along and says "Hey, this breaks feature "X").

The developers are taking a lot of care to try and ensure that the next version comes out bug-free. Wouldn't it be nice if we could provide an army of savvy testers to saturate the betas with potential problems, so they don't become problems in the release candidates or final release?

Your language needs you! Seriously ... and particularly if you run the Windows version!

Robert (R0ml) Lefkowitz at OSCON

There don't appear to be any methodologies that are specific to open source projects: methods like scrum are agnostic as to whether they are used on open source or proprietary software.

Lefkowitz has spent a long time trying to encourage large organizations to adopt open source techniques. His current employer uses the Microsoft solutions framework. This has a process model and a team model, and it's difficult even to get everyone in the organization to sign up to the models. Before the company used that they used another methodology with four phases, and Kent Beck's "Extreme Programming" uses six phases. All these methodologies are different, but subtly similar.

How do we define a methodology that everyone can use? R0ml believes we need to drive the development of the process broadly based in the open source world. He referred us to Quintillian's five-step rhetorical framework, which might have been lifted directly from the Microsoft framework had it not been written in the first century AD.

Rhetoric	MSF
inventio	Envision
dispositio	Plan
elocutio	Build
memoria	Stabilize
pronuntiato	Deploy

Other techniques or methodologies can also be mapped on to the rhetorical framework.

The architect Alexander observed that the events that take place in a place are primary in determining its fitness for purpose. Juggling uses two different notatiopositional notation, but some jugglers use "causal" notation, which focuses on throwing a club when it must be thrown to maintain stability. 70% of the MVS operating system is dedicated to exception handling, which is why they are so stable.

R0ml suggests starting with "release early/release often" as a starting point. Next, you use it (which you will notice is completely ignored by traditional methodologies). Bug and exceptions reports and patch submissions are then followed by triage and integration. The difference here is we don't have "requirements". There is no development, there is only maintenance; it's just that some maintenance is more radical that others (laughter).

XP puts some stress on the customer by requiring them to make decisions, write user stories, write functional test specifications, and so on. But the most powerful input from the customers is their complaints when the software doesn't do what they want. (More laughter).

This was an amusing talk, which anyone who has seen this speaker in action before would expect, but I'm not sure there were many pieces of practical advice for open source developers. I got the impression from a somewhat abrupt finish that Lefkowitz didn't have enough time to fully expound his thesis. A shame, since the last talk I saw was informative as well as amusing.

For some reason a number of posts were stuck in the blog as drafts. This one is from July last year.

Mark Shuttleworth at OSCON

Mark, an accomplished speaker, started by confessing that he had no doubt that Linux would be the platform of the future, and suggesting that tools like Firefox and bazaar had claimed ownership of their application space by providing extensibility in a modular way: conform to the interfaces, and the application becomes a platform. All the same, he insisted there is also a need to work with Windows (which makes sense since there are so many Windows systems around).

One of the issues Mark discussed was how we can extend agile techniques in ways inspired by community-driven processes. Architecting your tool set and your enterprise for collaboration and communication is essential. Ubuntu's governance is entirely separated from Canonical's, and Mark suspects that might turn out to be a best practice.

Tools must be extensible, and whether they are open source or proprietary they must be usable as components in diverse systems. The ultimate goal is "permissionless" development - new people coming in wanting to do new things need to be able to take the tools and run with them to implement their own ideas. His ideal for Ubuntu is for any developer worldwide to branch the Ubuntu code and publish their version in ways that other people can in turn pick up and run with.

Another best practice Mark has identified is time-based releases, which lend a rhythm to the development process and allow developers to keep track of what's happening in the trunk, which should be releasable at any time. More and more projects are thinking about synchronizing their releases. Imagine if a multitude of Linux distributions could collaborate to release at the same time, providing a "development cadence", encouraging individual tool developers to finalize their own releases for incorporation into a multitude of distributions.

Mark sees todays open source development world as struggling to provide free software, but the questions is how will the economic models support this? Free software is easy to love: it costs nothing to acquire. But who will fund the development of the free software? This will require innovations in economics that will, with a historical perspective, possibly be one of the more significant changes in these turbulent times. Having three almost-monopolies each controlling their own distribution would be counter-productive, and advertising on the desktop is offensive.

At Canonical they are hiring people to work on the desktop to build tools that benefit the providers of services. This will fundamentally change the world.

The challenge of the next two years is to lift the Linux desktop to a level where it is effectively art! How do Linux distributions not emulate Apple, but blow right past them. Jamie Zawinski says we should be aiming to build software tools that help their users get laid! The only doubts Mark had about that as a goal were related to Jamie's stated goal of making software that his mother could use ...

The Web 2.0 explosion has taught us that to survive in the web world a software product needs to be instantly attractive, and deliver functionality immediately in a profoundly usable way. Canonical have been investing in the process and the technology, and they are now trying to
realize the dream that Mark so elegantly elucidated in his presentation.

Django 1.0 Alpha 1 released

We learned yesterday that the Django development team had released the first alpha on the trail to the 1.0 release due early in September.

This is great news: it means that the major items that have been developed on the sidelines (a refactoring of the queryset code and a rewrite of the admin application to use NewForms) are now fully merged into the trunk. Most of the functionality for the eventual release is now in place, though of course the feature freeze doesn't start until the betas, scheduled for two weeks time.

OSCON Bound

Flying out to Portland tomorrow arriving late evening (I only just got back from California, and I wanted a little time at home). I am giving a Python in 3 Hours tutorial on Monday morning, but don't have many other set commitments during the week.

Anyone who wants to get together for a meal or a drink, just leave a comment here, or email me.

CPython Getting Serious About Quality: Help Needed

Beta2 of Python 2.6 and 3.0 was due out today. It isn't going to happen, and for the best of reasons. Under the guidance of release manager Barry Warsaw the developers have started to pay serious attention to the output of the buildbots, the systems which perform continuous integration testing of the development version of the Python interpreter.

Barry has just announced that there are three issue serious enough to delay the release, even though this potentially means slipping the release schedule (though time will tell whether that's a serious concern). If you are a Python developer, please take notice of what he said in an email to the python-dev list earlier today:

Please everyone, if you have only a little bit of time to work on Python, I hope you will attack the release critical and deferred blocker issues, and work on turning the buildbots green.  These are the top priorities in order to get 2.6 and 3.0 out on time.  And just as added incentive, our October 1st goal is being noted by downstream vendors. [Possibly confidential information about third parties redacted].

Quality is important. Please do whatever you can to help. Many people can benefit from a little extra effort now.

PyPy Continues to Catch Up

Interesting news from the EuroPython sprints at Vilnius via Holger Krekel. PyPy supports Django well enough to run the Pinax application put together by an open source team coordinated by James Tauber. It's great to see Python expanding in so many directions at once. The future is definitelyto be interesting.

Django seems to be becoming one of the "how good is your implementation" tests for Python implementations. Not only are the PyPy team using it in this way, but IronPython and Jython are as well. Since Jython is now in alpha with a Python 2.5-compatible implementation the picture is looking better and better every day.

The one thing nobody has discussed much is the attitude of the PyPy, Jython and IronPython development teams to migration towards Python 3.0. Does it represent the next wave, or will it end up as an isolated development?

Shameful Day in American History

So, it's official. Corporations are above the law, individual freedom should be subservient to corporate profits and if the president tells you to break the law well you can just go ahead and break it. The revolution moves a little nearer.

Certificate Design Required

Do you know anyone who can design a really nice looking certificate? Even better, are you such a person? The PSF instituted an awards scheme recently and the first four awards were announced at PyCon earlier this year. I have to have an official certificate created so we can have copies framed for the recipients, and my original designer has let me down (too busy).

So I am asking the web if it knows a suitable designer, preferably coming with recommendations, who can quote me a price (I can provide a reasonable brief if you contact me via this link) and get this job off my plate before the delay becomes too embarrassing.

July 8-10 Course Purchases

Please note: if you have attempted to purchase a place on our class on July 8-10, the purchase button was taking you to a page that showed the original (June) dates for the class. We have now changed this to agree with the current schedule, so you will see the correct dates on the purchase page. Please accept our apologies for any confusion.

No matter which button you used, if you have purchased a place on the class you should shortly receive a confirmation letter for your place on the class.

Help Shoot Down New Telco Immunity Threat

From the Electronic Frontier Foundation [italics added here]

ACTION ALERT: THE SHAM TELECOM IMMUNITY "COMPROMISE" IS LURKING IN THE WINGS, awaiting the signal for showtime. Though some members of Congress are guilty of selling the [rumored] bill's telecom immunity provision as a "compromise," there is still time to expose the telecom immunity proposal as a sham and to halt the bill before its too late. Already, some Congressional leaders have expressed concerns, and other influential members have expressed resistance to the scheme. Please join in and make your voice heard to all the members of Congress -- no false compromises for lawbreaking telecoms!
http://www.eff.org/action/no-false-compromises

I fully expect to see some shameful special-interest pandering by the exiting Bush administration as it breathes its weary last. Corporate America must learn to be responsible for its actions, even when it has broken the law at the President's request. Otherwise it will make foreign nations much more reluctant to house American enterprises. I seem to remember a time when "I was just obeying orders" wasn't accepted as an excuse for any kind of wrongdoing.

Why Twisted? - July 11, Washington, DC

Duncan McGreggor recently announced Holden Web's first public joint venture with Divmod but it struck me that our readerships may be quite disjoint, and so I should mention it too.

I know from Teach Me Twisted that there is a thirst for knowledge about this fascinating system and so I agreed to do something else about that, for a wider audience than PyCon. Consequently Duncan, Glyph and I agreed to follow our next public Introduction to Python class with a half-day seminar on how Twisted can create scalable networking solutions to real problems.

My own contribution will be inspired by Duncan's recent Mantissa: An Alternative to LAMP post, and will attempt to position Twisted in the marketplace. Glyph will be offering some really neat canned demonstrations of what Twisted can do [with a CD for managers to take back to their techies], and I am hoping that Duncan will be able to play the visionary role, taking us all into a future where networking is an essential component of all systems.

So, if you have a need to know about Twisted, and especially if you have budget, reserve a place now. There are only twelve places, and some are already taken.

We will give you two hours of presentations and follow that with a lunch where you can ask the speakers questions at leisure before departing to the remainder of your Friday afternoon. Sign up now!

Seth Excels Himself

Mostly this is a technical blog, but I feed a bit of marketing to the geeks now and then. I keep an eye on Seth Godin's blog, and this post caught my eye today. It ends with

The object isn’t to be perfect. The goal isn’t to hold back until you’ve created something beyond reproach. I believe the opposite is true. Our birthright is to fail and to fail often, but to fail in search of something bigger than we can imagine. To do anything else is to waste it all.

Amen to that. The person who never failed at anything never tried anything worthwhile.

"Introduction to Python" Class: July 8-10

Holden Web is pleased to announce its second public "Introduction to Python" class in the Washington, DC area. Intended for those with some programming background, the class gives you a working knowledge of the Python language reinforced by hands-on practical exercises. The class will be taught by Steve Holden, consultant, trainer and author of Python Web Programming.

If you know anyone who needs to learn Python (and who doesn't?), please point them to the registration page.

Divmod Partnership

Occasional readers may not remember how energized I was after my Teach Me Twisted session at PyCon this year, but I definitely wanted to follow it up with more. Audience members from that session will need no convincing that my ignorance of Twisted was almost total, but I am happy to say that it has now diminished a little. Determined effort on both sides has now yielded a training strategy for teaching other people Twisted too.

Twisted training will benefit from an analysis of the concepts that allows us to present them in the order required for specific understanding by a given client group. Divmod and Holden Web are jointly developing presentations that should make Twisted concepts approachable for a wider audience.

I am really looking forward to getting into some joint projects with the guys at Divmod. They have brains the size of planets. Stay tuned for an announcement about an introductory presentation.

Twisted Stakes a Claim

Disclosure: this post discusses remarks made by a Holden Web business partner.
In his Electric Duncan blog, Duncan McGreggor argues that Twisted now provides a viable LAMP alternative, and asks some very pertinent "so what?"-type questions:

As such, this is a direct competitor for LAMP. Here are some questions about that: What is the value of a full stack? Why is an alternative to LAMP good or needed? What is a good alternative? [...] We need to be asking ourselves what our applications are, what the network is, what services are, and what our dev teams and engineers need.

This led me to ask some questions of my own. Is this true? Is Twisted actually able to provide a full-service alternative web application solution? How adaptable is the architecture? Suppose I want to retain my Django templating instead of transitioning to Nevow: how easy would that be?

The case against LAMP starts with out-of-the-box scalability, as Sean Reifschneider (among other luminaries) has pointed out. I believe that Twisted has been proven at reasonably high levels. Whether it's processing Google-sized firehoses I don't know, but it's certainly supporting a significant part of the airline reservations structure.

As to "what is a good alternative", I suspect a lot of developers would go with "something that's orthogonal to my current code, so I don't have to rewrite anything". Perhaps that's asking a bit much, though. I know from various transitions I have made that some things just "fit my brain", and one of those was the Django templating scheme. Django users are pretty much database-agnostic, though a surprising number of them seem to use PostgreSQL. So perhaps Axiom support of Django templating?

The thing is, you can't just go around claiming to be an effective competitor to LAMP. You have to demonstrate the truth of that claim. I am sure the Twisted guys can do that if they are forced. They just don't realize the marketing necessity yet.

Nicholas Negroponte: An Ego Unbound?

I just finished reading Ivan Krstic's excellent Sic Transit Gloria Laptopi blog post. It appears that Negroponte would now like the OLPC XO to become yet another laptop we have to throw Windows off before we install some open source operating system supporting the GNU toolset.

I don't believe Nicholas Negroponte could manage to find his way out of a paper bag, let alone manage a hardware supplier trusted to build the infrastructure of developing countries. His Windows bait-and-switch makes him look like a shill for Microsoft. I wonder how much Microsoft stock he holds?

He just doesn't get that he has blown a huge chance to make a difference here. The thing that pisses me is the way he has played fast and loose with the work of literally hundreds of open source programmers who have attempted to support what they saw as a worthwhile educational project. But now Negroponte says that OLPC isn't an educational project after all. It's not about you. Nick, it's about the kids in the developing economies.

Thanks for the Money!

Well, apparently seven thousand other people also wanted to support brain tumor research yesterday at the eleventh annual "DC Race for Hope" in aid of the Brain Tumor Society and Accelerate Brain Cancer Cure. So many, in fact, that I didn't even pass the starting line until ten minutes after the first contestants. But it wasn't all about speed, which is just as well, since the lead runner past me on his return journey before I had even covered the first mile of the course.

The most important thing was the huge amount of money the event generated (a donation of a million dollars was announced just before the race commenced). Thanks once again to everyone who helped me to smash my sponsorship goal. With your help I raised $1,215, handsomely exceeding my target of $1,000, and my wildest expectations.

For those of you kind enough to inquire about my friend Sharon, her tumor turned out to be treatable with a combination of surgery and other therapies, and with luck she will be recovering from the surgery as I write this. I'm hoping that next year she can come to the race and join the other survivors. Thanks again for all your help!

The Media We Deserve?

Many thanks to Patrick Logan for bringing this New York Times article to my attention. It highlights the fact that the "military analysts" used (and paid!) by the major networks often appear to have business dealings and Pentagon relationships that cause blatant conflicts of interest with their independence as analysts.

My favorite quote shows that the networks feel they have something to hide, and in particular just exactly how "fair and balanced" Fox News wants to be:

CBS News declined to comment on what it knew about its military analysts’ business affiliations or what steps it took to guard against potential conflicts.

NBC News also declined to discuss its procedures for hiring and monitoring military analysts. The network issued a short statement: “We have clear policies in place to assure that the people who appear on our air have been appropriately vetted and that nothing in their profile would lead to even a perception of a conflict of interest.”

Jeffrey W. Schneider, a spokesman for ABC, said that while the network’s military consultants were not held to the same ethical rules as its full-time journalists, they were expected to keep the network informed about any outside business entanglements. “We make it clear to them we expect them to keep us closely apprised,” he said.

A spokeswoman for Fox News said executives “refused to participate” in this article.

CNN requires its military analysts to disclose in writing all outside sources of income. But like the other networks, it does not provide its military analysts with the kind of written, specific ethical guidelines it gives its full-time employees for avoiding real or apparent conflicts of interest.

Nothing like shutting up when you have something to hide. You can almost hear the ranks closing.

Microsoft Can't Support DRM

So it seems that "PlaysForSure" actually meant "Plays until the first time you change your computer or operating system after August 31, 2008". About ten days ago Microsoft's general manager for MSN Entertainment and Video Services sent out an email to customers advising them that license keys can no longer be retrieved and new computers cannot be activated after that date.

This is a very clear indication that digital rights management (DRM) is a loss for the consumer. If I wanted to keep songs from my eight-track stereo around then there was nothing to stop me from transferring them to audio cassette or CD (despite the RIAA's rather specious arguments that this was a breach of copyright). If I'd been silly enough to trust Microsoft's assertion that DRM offered me protections then I'd have to get ready to shell out for new copies after any change to my playing environment.

The Electronic Freedom Foundation has written to Microsoft stating in no uncertain terms that (by suggesting that users copy the music to a CD and transfer it to new computers that way) the company "is asking its customers to invest more time, labor and money in order to continue to enjoy the music for which they have already paid".

If Microsoft, with all their resources, can't keep a DRM scheme working then there is little chance for anyone else. It also doesn't encourage customers to trust the company if it is prepared to abandon them in this way, though they might have anticipated this desertion when Microsoft closed the MSN store when it started selling the Zune.

I suspect this is the beginning of the end of DRM. Let it rest in peace.

Twisty Little Passages ...

I have been rather disappointed that the "Teach Me Twisted" session I ran at PyCon didn't get more heavily blogged. Not only did I have a lot of fun, but I also learned something. The Twisted guys learned something too, I'm happy to say.

No announcements yet, but with luck it will eventually lead to bigger things. That session may have been a butterfly flapping its wings.

Django Nudges Closer to 1.0

Malcom Tredinnick just announced that he has merged the long-awaited queryset-refactor branch into the Django trunk. This puts one of the major blocks in place for the 1.0 release.

James Bennett recently posted his list of requirements for a 1.0 release (with a warning that suggestions from twits would not be tolerated: do not annoy this man if you don't know what you are talking about), and this announcement reduces the length of the list to eleven items.

So it's got significantly closer, as queryset-refactor was a biggie. Now I have to go test my code against the new branch ...

Impressive Silly Trick

Nice to see that the Python community still enjoys a challenge!

I Want Your Money!

Not for myself, I hasten to add, but to support the Brain Tumor Society. The Race for Hope 5km run is an annual event organized by my wife's employer, and we had agreed to participate before we discovered recently that one of our friends has a brain tumor. It's too early to know whether it's operable at this stage, so we are hoping for the best.

[UPDATE 4/22: I have discovered that my wife has already raised almost $2,500 so I have upped my target somewhat to see if we can't show the world that geeks aren't cheap!]

In the meantime, I have set what I feel is a fairly modest goal for additional sponsorship. I am hoping readers will show how generous the open source world can be. If you've ever enjoyed a PyCon then please drop by my personal race page to make a donation, and point your friends to it as well. I'll be quite happy to up the target if the existing one proves too easy, and any amount will be welcome.

Thanks to anyone generous enough to support this effort. If the team I'm on is in the top 5 fund raisers next week we'll have an extra $1,000 added to our sponsorship, so dig deep and treat the target as something to beat, not exceed.

Impeach?

I mostly keep politics (except perhaps the politic of open source) out of this blog, so if you aren't interested you won't want to follow this link. I'm not sure why I got so fired up about this today, specifically, except that the day's news has been full of politics. But when wasn't it, lately?

Blogger Annoyances #319

Is this just me, or is there some trick I just haven't learned? I was reminded, yet again, when completing the previous post in this blog, that Blogger tagging is way less user-friendly and convenient than del.icio.us and other similar sites.

I wanted to tag the item with "mono", but alas I was foolish enough to tag a previous post with "monopoly". So when I enter "mono" the widget is already suggesting the auto-completion (see image), and when I type a comma, it completes the entry as "monopoly", the existing tag. I try entering an escape character—no good, the comma still completes it as the existing tag, as does a tab.



Eventually I let it complete as "monopoly" and then go back and delete the last four characters from the tag. I really dislike applications that insist they know better than me. Since tab expands the tag to complete as an existing tag, why the heck does a comma have to do the same thing? The real annoyance is: I have this ugly feeling that somewhere out there there's a programmer feeling really smart for having committed this atrocity. Someone, please tell me there's a sensible way to do this before I go mad.

Learning IronPython

Some good advice from Curt Hagenlocher on the IronPython mailing list in response to this question:

I program in Python, but my programs are typically run from the command line or IDLE. I have never built a GUI, and would like to try my hand at creating GUIs that I can run my programs from. Is there a book or tutorial that covers IronPython, IronPython Studio and the dot Net platform that is aimed squarely at the new user?

Any book or tutorial about the .NET platform in general will be relevant to IronPython's use of .NET. In fact, taking existing C# or VB examples and translating them into Python may be a good way to leverage your existing Python skills while teaching yourself the .NET class libraries.

As far as IronPython-specific resources go, you can't beat the IronPython Cookbook at http://www.ironpython.info/index.php/Main_Page. If you want to see some sample Python code that uses the Windows Forms or Windows Presentation Foundation libraries to produce a GUI, just go to the Contents at http://www.ironpython.info/index.php/Contents and scroll down a bit.

It's fashionable to pooh-pooh .NET, but remember that this stuff runs on Mono too, so it could be an important way to build cross-platform compatibility in Python applications. Not everything that came out of Redmond is worth ignoring (but let's not talk about OOXML), so there's possibly something to be gained from a knowledge of that framework. Don't throw the baby out with the bathwater.

Galcon Cheats!

I'm not going to ask for my money back, or anything, but I really do think it's a bit naughty of Galcon to mobilize its forces before I've even pressed the start button.

If you haven't tried this game by Phil Hassey (met him at PyCon, terrific bloke) it's quite an addictive space war adventure where the objective is to overrun neutral and opposing forces to gain control of planetary territory.

You can also run it as a network server if you want to host multi-player games! Well worth a try. But it still cheats, dammit.

For the Want of a Nail

We've all done it. One step forward, two steps back. I just thought I'd try to relieve my current frustrations by writing this particular sequence up. It's amazing the diversions and byroads you can end up taking to achieve an apparently simple goal (as I learned yet again last week in trying to accommodate Outlook on my SMTP server). This one started at PyCon, now four weeks in the past. So, bear in mind the goal: submit a Django documentation change.

At the Django sprint (of which, due to a family illness, I was only able to attend a truncated first day) I was struggling with an unstable laptop, so I managed to assign myself a documentation ticket, and that turned out to be pretty much it what with PSF responsibilities and needing to go home early. I have been working on the ticket since: it isn't a difficult issue, but I needed to learn a little more about how the Django database layers worked.

Having finally decided on the required change, during entry in the Django tracker I decided it would be much more developer-friendly if I were to provide the change in the form of a patch. This isn't really necessary but I figured it couldn't hurt, and that I should really be able to rebuild the documentation locally to be sure that any future changes were stylistically correct and didn't mangle what is, after all, quite good documentation.

This is where things started to get a little rough. Descending into the documentation directory I discover that it's driven by a Makefile. "No problem", I think, "I have Cygwin, I can build under that". So I type "make html" and discover I don't have make installed. This is quite normal, I like to keep Cygwin fairly lean, and I haven't previously done any Cygwin building on the new laptop.

So, I run the usually very cooperative Cygwin installation utility, and discover that my usual mirror site is providing very slow response. So slow, in fact [minor pause here to repair cable retraction mechanism on traveling mouse: this really is not my day] that several attempts to update my installation have to be abandoned, though it notes that several of my other installed applications should also be upgraded and has even managed to pull down some of the installers.

Eventually the fifth or sixth attempt gives me the dread message from Vista (how I hate this operating system, yet despite my best efforts there are tasks I currently still need it for) that the Cygwin installer "is not working correctly".

After a reboot and several further repeats, I eventually track this down to a corrupted /etc/setup/openshh.lst.gz file, and delete it. I establish contact with a different mirror and lo! The Cygwin installer now runs to completion, and I can run make. So I confidently re-enter "make html" and see the message

make: sphinx-build: Command not found

Fair enough, it shouldn't (I foolishly think, despite all the evidence that should by now have convinced me to set the computer aside for the day) take me long to get that up. As a first stab, before doing anything else, I type "easy_install sphinx-build". Hello, another even more interesting error message:

bash: /c/Python25/Scripts/easy_install: Permission denied

What? ls -l gives me the following rather unsavory news:

----------+ 1 sholden None 6656 Apr 2 10:33 /c/Python25/Scripts/easy_install

Damn, that can't be right? How did that happen? Well by this time I am past caring how it happened. I own the file, so I'll just reassert the correct permission, right? Wrong:

$ chmod 755 /c/Python25/Scripts/easy_install chmod: changing permissions of `/c/Python25/Scripts/easy_install': Permission denied

Now this is getting really annoying. Note how my problem has subtly changed from "how do I submit a Django documentation change" to "how do I get Cygwin to allow me to change the permissions on this file", and that I am now several steps further away from my original goal than I appeared to be at that happy moment when I set out to submit the change. Note also the assumption that running easy_install is actually necessary to solve my problem. If this had turned out to be wrong I would probably have shot myself, thereby relieving you of the task of reading this post.

Fortunately I quickly realize that this is the easy_setup from my Windows Python that I am trying to run, and that the solution is to run ez_setup.py with my Cygwin Python. [There are some who doubt the wisdom of doing this to a Python installation, but I am blasé beyond belief about such things]. So I run ez_setup.py, which installs setuptools. There is the minor annoyance that Cygwin refuses to recognize the correct version of easy_install and continues to run the Windows copy, but I solve that by opening a new Cygwin terminal window. Another final false start: the package required isn't called sphinx-build, so easy_setup can't load it. So I take another reasonable guess.

Now, finally, I run easy_install sphinx followed by make html, and in less time than it take to write a blog entry I have a local copy of the documentation that I can patch to my heart's content. Wonderful things, computers. When they do what you want.

Tweaking the Bytecode

Ned Batchelder has come up with an interesting hack for increasing the granularity of his code coverage testing. It involves modifying the code object associated with the code being modified: specifically, the co_lnotab member contains a starting line number followed by a sequence of (code offset, line number delta) pairs.

Ned's technique involves rewriting the member so that each bytecode appears to be on a single line on its own. While this allows him to ensure that all bytecodes are exercised, he hasn't yet ironed out the quirks in how to use that to present coverage information (which is hardly surprising, since most Python programmers could care less about the structure of their bytecode).

It seems to me, however, that by mapping the modified co_lnotab back to the original you could at least flag lines whose code isn't 100% covered by your tests, and this is surely valuable information. Ned's already thought of this, and includes the throwaway remark "The original line number information is lost. It would have to be stored off to the side to make this useful" towards the end of his blog entry.

So I believe we can expect coverage.py to start producing even more useful information soon.

Meme du Jour

Interesting. I was quite surprised by this myself.

# history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}' |sort -rn|head
135 svn
134 ls
87 vi
76 postmap
57 cd
45 more
36 ps
35 /etc/init.d/httpd
33 /usr/sbin/postfix
32 grep

This is on my (virtual private) web server, where I've been editing fairly heavily recently.

Outlook is Driving Me Nuts!

[UPDATE: two changes were required. First I had to set broken_sasl_auth_clients = yes in my postfix main.cf, then I had to add login as an available authentication mechanism in dovecot, which postfix uses as an authentication server. Thanks for all pointers and suggestions, some of which came by e-mail. Not quite as stupid and fundamental as I feared. Steve].

OK, I am in serious need of assistance here. I suspect that there must be something really stupid and fundamental that I am overlooking. The problem is that I cannot get Outlook 2003 to authenticate to its SMTP server.

I have tried deleting the account and recreating it (shutting Outlook down in between). I have verified that the same settings in a Thunderbird client allow me to send mail without difficulty. I have even installed Outlook on an entirely separate computer and got the same dismal results.

Why Outlook? Well, although I am a Thunderbird user myself my wife uses Outlook. So my most important support client has a problem -- only with sending mail, though, receiving works fine.

Just so you can visually verify that I haven't done anything stupid, here are screen shots of the relevant dialogs. I have even set up a local printing SMTP proxy to verify what's happening! With Thunderbird it clearly shows an AUTH command being send to the server, but with good 'ole Outlook I get what you see below (slightly redacted): straight from EHLO to MAIL with no AUTH at all.

Connected
==> (0) '220 holdenwe.user.openhosting.com ESMTP Postfix\r'
<== (0) 'EHLO DoroPC\r' ==> (0) '250-holdenwe.user.openhosting.com\r'
==> (0) '250-PIPELINING\r'
==> (0) '250-SIZE 10240000\r'
==> (0) '250-VRFY\r'
==> (0) '250-ETRN\r'
==> (0) '250-AUTH PLAIN\r'
==> (0) '250-ENHANCEDSTATUSCODES\r'
==> (0) '250-8BITMIME\r'
==> (0) '250 DSN\r'
<== (0) 'MAIL FROM: <xxxx@holdenweb.com>\r'
==> (0) '250 2.1.0 Ok\r'
<== (0) 'RCPT TO: <xxxx@gmail.com>\r'
==> (0) '554 5.7.1 <xxxx@gmail.com>: Relay access denied\r'
<== (0) 'QUIT\r' 

So, if you can tell me what's wrong you may rescue my sanity! Help me, please, this is driving me nuts.

Google Busts the Web Wide Open - In Python!

Well, if ever there was a compelling reason for using Python it's probably Google App Engine. Here's the canonical "hello world" Google App:

print 'Content-Type: text/plain'
print ''
print 'Hello, world!'

Doesn't get a lot simpler than that, does it?

There's a lot behind this announcement, including a detailed description of the data store (which interestingly seems to bear the same kind of relationship to a classical relational database that Python does to statically-typed languages), access to some of the attributes of authenticated Google account users, the ability to fetch remote URLs and a startlingly straightforward way to send email.

If I were going to write another book about Python web programming I wouldn't be able to ignore this development. What a tempting idea ...

Twisted Bite the Web Bullet

The Twisted community has decided it's time to clean up its web act. The problem for a while has been that there were three "competing" packages that might have formed the basis of any web application: twisted.web, nevow and twisted.web2.

Now they have decided that twisted.web is the right namespace, and the immediate plan is to backport the stable functionality from web2 and the nevow application server into twisted.web, giving a more advanced set of web functionality in a way that offers best compatibility to existing users. Nevow will intially remain, as a pure templating solution, but there's a possibility that functionality might also be repositioned eventually.

This has the smell of a compromise, but as compromises go (and as you'd expect in the Twisted world) it seems to be a pragmatic engineering solution to the current rather confusing situation.