Twisty Little Passages ...

I have been rather disappointed that the "Teach Me Twisted" session I ran at PyCon didn't get more heavily blogged. Not only did I have a lot of fun, but I also learned something. The Twisted guys learned something too, I'm happy to say.

No announcements yet, but with luck it will eventually lead to bigger things. That session may have been a butterfly flapping its wings.

Django Nudges Closer to 1.0

Malcom Tredinnick just announced that he has merged the long-awaited queryset-refactor branch into the Django trunk. This puts one of the major blocks in place for the 1.0 release.

James Bennett recently posted his list of requirements for a 1.0 release (with a warning that suggestions from twits would not be tolerated: do not annoy this man if you don't know what you are talking about), and this announcement reduces the length of the list to eleven items.

So it's got significantly closer, as queryset-refactor was a biggie. Now I have to go test my code against the new branch ...

Impressive Silly Trick

Nice to see that the Python community still enjoys a challenge!

I Want Your Money!

Not for myself, I hasten to add, but to support the Brain Tumor Society. The Race for Hope 5km run is an annual event organized by my wife's employer, and we had agreed to participate before we discovered recently that one of our friends has a brain tumor. It's too early to know whether it's operable at this stage, so we are hoping for the best.

[UPDATE 4/22: I have discovered that my wife has already raised almost $2,500 so I have upped my target somewhat to see if we can't show the world that geeks aren't cheap!]

In the meantime, I have set what I feel is a fairly modest goal for additional sponsorship. I am hoping readers will show how generous the open source world can be. If you've ever enjoyed a PyCon then please drop by my personal race page to make a donation, and point your friends to it as well. I'll be quite happy to up the target if the existing one proves too easy, and any amount will be welcome.

Thanks to anyone generous enough to support this effort. If the team I'm on is in the top 5 fund raisers next week we'll have an extra $1,000 added to our sponsorship, so dig deep and treat the target as something to beat, not exceed.

Impeach?

I mostly keep politics (except perhaps the politic of open source) out of this blog, so if you aren't interested you won't want to follow this link. I'm not sure why I got so fired up about this today, specifically, except that the day's news has been full of politics. But when wasn't it, lately?

Blogger Annoyances #319

Is this just me, or is there some trick I just haven't learned? I was reminded, yet again, when completing the previous post in this blog, that Blogger tagging is way less user-friendly and convenient than del.icio.us and other similar sites.

I wanted to tag the item with "mono", but alas I was foolish enough to tag a previous post with "monopoly". So when I enter "mono" the widget is already suggesting the auto-completion (see image), and when I type a comma, it completes the entry as "monopoly", the existing tag. I try entering an escape character—no good, the comma still completes it as the existing tag, as does a tab.



Eventually I let it complete as "monopoly" and then go back and delete the last four characters from the tag. I really dislike applications that insist they know better than me. Since tab expands the tag to complete as an existing tag, why the heck does a comma have to do the same thing? The real annoyance is: I have this ugly feeling that somewhere out there there's a programmer feeling really smart for having committed this atrocity. Someone, please tell me there's a sensible way to do this before I go mad.

Learning IronPython

Some good advice from Curt Hagenlocher on the IronPython mailing list in response to this question:

I program in Python, but my programs are typically run from the command line or IDLE. I have never built a GUI, and would like to try my hand at creating GUIs that I can run my programs from. Is there a book or tutorial that covers IronPython, IronPython Studio and the dot Net platform that is aimed squarely at the new user?

Any book or tutorial about the .NET platform in general will be relevant to IronPython's use of .NET. In fact, taking existing C# or VB examples and translating them into Python may be a good way to leverage your existing Python skills while teaching yourself the .NET class libraries.

As far as IronPython-specific resources go, you can't beat the IronPython Cookbook at http://www.ironpython.info/index.php/Main_Page. If you want to see some sample Python code that uses the Windows Forms or Windows Presentation Foundation libraries to produce a GUI, just go to the Contents at http://www.ironpython.info/index.php/Contents and scroll down a bit.

It's fashionable to pooh-pooh .NET, but remember that this stuff runs on Mono too, so it could be an important way to build cross-platform compatibility in Python applications. Not everything that came out of Redmond is worth ignoring (but let's not talk about OOXML), so there's possibly something to be gained from a knowledge of that framework. Don't throw the baby out with the bathwater.

Galcon Cheats!

I'm not going to ask for my money back, or anything, but I really do think it's a bit naughty of Galcon to mobilize its forces before I've even pressed the start button.

If you haven't tried this game by Phil Hassey (met him at PyCon, terrific bloke) it's quite an addictive space war adventure where the objective is to overrun neutral and opposing forces to gain control of planetary territory.

You can also run it as a network server if you want to host multi-player games! Well worth a try. But it still cheats, dammit.

For the Want of a Nail

We've all done it. One step forward, two steps back. I just thought I'd try to relieve my current frustrations by writing this particular sequence up. It's amazing the diversions and byroads you can end up taking to achieve an apparently simple goal (as I learned yet again last week in trying to accommodate Outlook on my SMTP server). This one started at PyCon, now four weeks in the past. So, bear in mind the goal: submit a Django documentation change.

At the Django sprint (of which, due to a family illness, I was only able to attend a truncated first day) I was struggling with an unstable laptop, so I managed to assign myself a documentation ticket, and that turned out to be pretty much it what with PSF responsibilities and needing to go home early. I have been working on the ticket since: it isn't a difficult issue, but I needed to learn a little more about how the Django database layers worked.

Having finally decided on the required change, during entry in the Django tracker I decided it would be much more developer-friendly if I were to provide the change in the form of a patch. This isn't really necessary but I figured it couldn't hurt, and that I should really be able to rebuild the documentation locally to be sure that any future changes were stylistically correct and didn't mangle what is, after all, quite good documentation.

This is where things started to get a little rough. Descending into the documentation directory I discover that it's driven by a Makefile. "No problem", I think, "I have Cygwin, I can build under that". So I type "make html" and discover I don't have make installed. This is quite normal, I like to keep Cygwin fairly lean, and I haven't previously done any Cygwin building on the new laptop.

So, I run the usually very cooperative Cygwin installation utility, and discover that my usual mirror site is providing very slow response. So slow, in fact [minor pause here to repair cable retraction mechanism on traveling mouse: this really is not my day] that several attempts to update my installation have to be abandoned, though it notes that several of my other installed applications should also be upgraded and has even managed to pull down some of the installers.

Eventually the fifth or sixth attempt gives me the dread message from Vista (how I hate this operating system, yet despite my best efforts there are tasks I currently still need it for) that the Cygwin installer "is not working correctly".

After a reboot and several further repeats, I eventually track this down to a corrupted /etc/setup/openshh.lst.gz file, and delete it. I establish contact with a different mirror and lo! The Cygwin installer now runs to completion, and I can run make. So I confidently re-enter "make html" and see the message

make: sphinx-build: Command not found

Fair enough, it shouldn't (I foolishly think, despite all the evidence that should by now have convinced me to set the computer aside for the day) take me long to get that up. As a first stab, before doing anything else, I type "easy_install sphinx-build". Hello, another even more interesting error message:

bash: /c/Python25/Scripts/easy_install: Permission denied

What? ls -l gives me the following rather unsavory news:

----------+ 1 sholden None 6656 Apr 2 10:33 /c/Python25/Scripts/easy_install

Damn, that can't be right? How did that happen? Well by this time I am past caring how it happened. I own the file, so I'll just reassert the correct permission, right? Wrong:

$ chmod 755 /c/Python25/Scripts/easy_install chmod: changing permissions of `/c/Python25/Scripts/easy_install': Permission denied

Now this is getting really annoying. Note how my problem has subtly changed from "how do I submit a Django documentation change" to "how do I get Cygwin to allow me to change the permissions on this file", and that I am now several steps further away from my original goal than I appeared to be at that happy moment when I set out to submit the change. Note also the assumption that running easy_install is actually necessary to solve my problem. If this had turned out to be wrong I would probably have shot myself, thereby relieving you of the task of reading this post.

Fortunately I quickly realize that this is the easy_setup from my Windows Python that I am trying to run, and that the solution is to run ez_setup.py with my Cygwin Python. [There are some who doubt the wisdom of doing this to a Python installation, but I am blasé beyond belief about such things]. So I run ez_setup.py, which installs setuptools. There is the minor annoyance that Cygwin refuses to recognize the correct version of easy_install and continues to run the Windows copy, but I solve that by opening a new Cygwin terminal window. Another final false start: the package required isn't called sphinx-build, so easy_setup can't load it. So I take another reasonable guess.

Now, finally, I run easy_install sphinx followed by make html, and in less time than it take to write a blog entry I have a local copy of the documentation that I can patch to my heart's content. Wonderful things, computers. When they do what you want.

Tweaking the Bytecode

Ned Batchelder has come up with an interesting hack for increasing the granularity of his code coverage testing. It involves modifying the code object associated with the code being modified: specifically, the co_lnotab member contains a starting line number followed by a sequence of (code offset, line number delta) pairs.

Ned's technique involves rewriting the member so that each bytecode appears to be on a single line on its own. While this allows him to ensure that all bytecodes are exercised, he hasn't yet ironed out the quirks in how to use that to present coverage information (which is hardly surprising, since most Python programmers could care less about the structure of their bytecode).

It seems to me, however, that by mapping the modified co_lnotab back to the original you could at least flag lines whose code isn't 100% covered by your tests, and this is surely valuable information. Ned's already thought of this, and includes the throwaway remark "The original line number information is lost. It would have to be stored off to the side to make this useful" towards the end of his blog entry.

So I believe we can expect coverage.py to start producing even more useful information soon.

Meme du Jour

Interesting. I was quite surprised by this myself.

# history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}' |sort -rn|head
135 svn
134 ls
87 vi
76 postmap
57 cd
45 more
36 ps
35 /etc/init.d/httpd
33 /usr/sbin/postfix
32 grep

This is on my (virtual private) web server, where I've been editing fairly heavily recently.

Outlook is Driving Me Nuts!

[UPDATE: two changes were required. First I had to set broken_sasl_auth_clients = yes in my postfix main.cf, then I had to add login as an available authentication mechanism in dovecot, which postfix uses as an authentication server. Thanks for all pointers and suggestions, some of which came by e-mail. Not quite as stupid and fundamental as I feared. Steve].

OK, I am in serious need of assistance here. I suspect that there must be something really stupid and fundamental that I am overlooking. The problem is that I cannot get Outlook 2003 to authenticate to its SMTP server.

I have tried deleting the account and recreating it (shutting Outlook down in between). I have verified that the same settings in a Thunderbird client allow me to send mail without difficulty. I have even installed Outlook on an entirely separate computer and got the same dismal results.

Why Outlook? Well, although I am a Thunderbird user myself my wife uses Outlook. So my most important support client has a problem -- only with sending mail, though, receiving works fine.

Just so you can visually verify that I haven't done anything stupid, here are screen shots of the relevant dialogs. I have even set up a local printing SMTP proxy to verify what's happening! With Thunderbird it clearly shows an AUTH command being send to the server, but with good 'ole Outlook I get what you see below (slightly redacted): straight from EHLO to MAIL with no AUTH at all.

Connected
==> (0) '220 holdenwe.user.openhosting.com ESMTP Postfix\r'
<== (0) 'EHLO DoroPC\r' ==> (0) '250-holdenwe.user.openhosting.com\r'
==> (0) '250-PIPELINING\r'
==> (0) '250-SIZE 10240000\r'
==> (0) '250-VRFY\r'
==> (0) '250-ETRN\r'
==> (0) '250-AUTH PLAIN\r'
==> (0) '250-ENHANCEDSTATUSCODES\r'
==> (0) '250-8BITMIME\r'
==> (0) '250 DSN\r'
<== (0) 'MAIL FROM: <xxxx@holdenweb.com>\r'
==> (0) '250 2.1.0 Ok\r'
<== (0) 'RCPT TO: <xxxx@gmail.com>\r'
==> (0) '554 5.7.1 <xxxx@gmail.com>: Relay access denied\r'
<== (0) 'QUIT\r' 

So, if you can tell me what's wrong you may rescue my sanity! Help me, please, this is driving me nuts.

Google Busts the Web Wide Open - In Python!

Well, if ever there was a compelling reason for using Python it's probably Google App Engine. Here's the canonical "hello world" Google App:

print 'Content-Type: text/plain'
print ''
print 'Hello, world!'

Doesn't get a lot simpler than that, does it?

There's a lot behind this announcement, including a detailed description of the data store (which interestingly seems to bear the same kind of relationship to a classical relational database that Python does to statically-typed languages), access to some of the attributes of authenticated Google account users, the ability to fetch remote URLs and a startlingly straightforward way to send email.

If I were going to write another book about Python web programming I wouldn't be able to ignore this development. What a tempting idea ...

Twisted Bite the Web Bullet

The Twisted community has decided it's time to clean up its web act. The problem for a while has been that there were three "competing" packages that might have formed the basis of any web application: twisted.web, nevow and twisted.web2.

Now they have decided that twisted.web is the right namespace, and the immediate plan is to backport the stable functionality from web2 and the nevow application server into twisted.web, giving a more advanced set of web functionality in a way that offers best compatibility to existing users. Nevow will intially remain, as a pure templating solution, but there's a possibility that functionality might also be repositioned eventually.

This has the smell of a compromise, but as compromises go (and as you'd expect in the Twisted world) it seems to be a pragmatic engineering solution to the current rather confusing situation.

One-Day Class Topics

Since the first Introduction to Python class seemed to go quite well I am thinking about expanding the offerings by slotting in one-day introductory classes on Python-related topics.

From PyCon I know there is a lot of interest in both Twisted and Django. But I wonder how to contact the people who want to learn these technologies. Are they already members of the Python community (to whom this blog is primarily addressed most of the time), or do I need to generate some new marketing channel to reach them?

What other Python technologies do people need a fast introduction to?

Just Because I Can?

Seth Godin as usual makes some interesting points in a blog entry today, including two very salient ones for me:

a) blogging doesn't have to be perfect, so blog and move on rather than waiting for the perfect way to express a transient thought;
b) you don't express any ideas at all by not writing, so writing a little and often may well be more useful than producing large well-crafted essays every six months.

The first is a message that has started to get through to me anyway as a result of my work at Python Magazine; the second is a variation on the theme that nobody knows what history will make of them, so there's little point trying for a particular effect.

Thanks for getting me started again Seth!