The SuperVillain Test

Happy 2007, everyone!
Oh, no, what a way to start a new year. I am Lex Luthor

Lex Luthor	62%
Dr. Doom	57%
Kingpin	55%
Green Goblin	52%
Mr. Freeze	46%
The Joker	39%
Apocalypse	34%
Riddler	33%
Poison Ivy	30%
Catwoman	29%
Mystique	28%
Magneto	27%
Venom	21%
Dark Phoenix	12%
Juggernaut	12%
Two-Face	12%

A brilliant businessman on a quest for world domination and the self-proclaimed greatest criminal mind of our time!

Click here to take the "Which Super Villain are you?" quiz...

Then again, looking at the other options maybe Lex Luthor isn't so bad ... at least the guy has some charisma. Overall I suspect I'm just not really supervillain material.

The SuperHero Test

I saw this on Allessandro Iob's blog. I too am Iron Man. Maybe I should start using IronPython?


Your results:
You are Iron Man

Iron Man 80% Spider-Man 80% Robin 72% Hulk 70% Superman 65% The Flash 60% Supergirl 55% Catwoman 50% Wonder Woman 40% Green Lantern 40% Batman 35%

Inventor. Businessman. Genius.

Click here to take the "Which Superhero am I?" quiz...

W'appen?

I see Blogger have released their beta developments into production. I can't help wondering if that's why my previous posting doesn't seem to have made it through RSS feeds and into the places it usually appears. It's not every day something so noteworthy happens.

Noteworthy?

Well, thanks to coralpoetry for pointing out this has become a blog of note, presumably the last of 2006. I suppose this might be more humbling if there were published criteria for what makes a blog noteworthy, or if I had been aware of this feature before it linked to ... for Some Value of Magic. Another ten seconds of my fifteen minutes of fame?

Acknowledgements of any sort are encouraging, though. Like the old saying about bad publicity, for a minor blog there's no such thing as unwelcome atttention! So many thanks to whoever at Blogger (Google) put me on that list and to anyone else who happens to read this blog.

Because of the linkage I suspect that a small spike in readership is likely to occur, so let me take this opportunity to wish all fellow-members of the blogosphere (dreadful word) and all Holden Web customers a happy and prosperous new year. 2006 (and especially the latter part of it) has been somewhat traumatic for various reasons, so here's hoping for a smoother 2007.

Finally, while I have your attention, let me point you to Pass IT On, my favourite charity. Martha and Sandy's dedication to their goal of providing computer equipment to the disabled is single-minded, and I wish I could do more to help them. If Santa Claus has brought you a new computer please consider donating your old one. They also need cash support to help them become a free-standing organisation this year, so you can give them money if you don't have a computer to spare. Good luck to all at Pass IT On.

ZFS: "Probably more abuse in 20 seconds than you'd see in a lifetime"

I've been reading (thanks to Brett Cannon) about the Zettabyte File System (ZFS) - an open source project from Sun Microsystems that seems to offer real opportunities to take proper advantage of both SAN and virtualization technologies. If you want to know what a zettabyte is then as usual the Wikipedia is reliable on technical matters (and no, I had never run across the term before).

Brett was simply happy that Apple announced the inclusion of ZFS into the Leopard (OS X 10.5) distribution. I was interested because Sun developers were taking an interest in advanced filesystems even back when I used to work there - they produced the Translucent File System (TFS), which was a copy-on-write mechanism to allow users to share a common source pool and keep their own changes.

One of the problems with TFS was its need for kernel integration (in those days even the windowing system was integrated into the kernel). Another issue arose when a user base their TFS on another TFS which was based on a third TFS which ... effectively it implemented an copy-on-write inheritance mechanism, but efficiency could drop quite rapidly with multiple layers.

Unlike TFS, ZFS can run in either kernel or user mode, and the nightly testing appears to twist the filesystem in every possible direction simultaneously (hence the title of this blog entry). Here's a list of what goes on in parallel in each nightly test:

? Read, write, create, and delete files and directories
? Create and destroy entire filesystems and storage pools
? Turn compression on and off (while filesystem is active)
? Change checksum algorithm (while filesystem is active)
? Add and remove devices (while pool is active)
? Change I/O caching and scheduling policies (while pool is active)
? Scribble random garbage on one side of live mirror to test self-healing data
? Force violent crashes to simulate power loss, then verify pool integrity

So, it would appear that ZFS is a worthy successor to TFS and much more besides, as you will find out if you too read the slide show. I haven't looked at the license yet, but it does appear that under Johnathan Schwartz's direction Sun might be beginning to find its way in the open source world of the 21st century, despite the amusing graphic that Eric Bangeman produced when Scott McNealy resigned. Maybe I should buy some stock again. Maybe we all should.

Lies, Damned Lies and Baggage Claim Call Centres

I travel quite a lot, and up until now I have been lucky. Bags have been lost from time to time, but they have always made their way back to me, usually within 24 hours. I understand that bags can get lost occasionally, but the frustrating thing last time this happened was the complete absence of any reliable information.

I flew from Edinburgh to London Heathrow, then onwards to Washington Dulles where I was incorrectly instructed to enter the USA through the transit lounge. Since my flight to New York's La Guardia was on a separate reservation and my bag had only been checked through to Washington this put me in the wrong place to reclaim it, and trriggered what I would call a comedy of errors if there was anything the least bit funny about it. My bag and I were both in Dulles, but United were unable to reunite us. I was initially assured that the bag would join me on the flight to La Guardia but, inevitably, I arrived and it didn't.

So I registered a claim and went to my accommodation. It took forty-eight hours and ten 'phone calls before I was finally able to wear the clothes I had brought with me. For most of that time United's web site was telling me that the bag had been found and that it would be sent to New York on a non-existent flight shortly before I landed at Washington!

Talking to other travellers, the general impression is that baggage "help" lines are uniformly bad. My own experience was certainly terrible: I could easily have believe that there was no computer-based information system at all behind the scenes, as the different people I spoke to gave me completely different and inconsistent stories each time I called. The responses almost seemed calculated to inflame. I was particularly incensed by a conversation with one woman who, when I called back as requested after waiting two hours, insisted that I had only called thirty minutes previously, and that I should wait until next morning before calling again. It's not usually considered good customer service to call the customer a liar. The different stories I was told about my luggage might as well have been chosen at random.

Under these circumstances the airlines would do better to dispense with the call centres completely and replace them with a recording saying "Our staff are working to restore your lost bag. Thank you for your patience". This would be less irritating than the nonsense I had to put up with, would have taken up far less of my time, and would hopefully reduce costs and allow the airlines to invest a bit more in relocating and delivering lost bags.

And now, if you'll excuse me, I have to send a claim for reimbursement to United.

Blogger on a Nokia 770

Well, I find myself with only a Nokia 770 Internet tablet to access the 'Net, so I thought I'd see how easy it was to create a blog entry. The typing is less than terrific through the hunt-and-peck on-screen keyboard, but so far I haven't been able to get my snazzy laser-projection VKB virtual keyboard to work reliably with the Nokia, so I have to be satisfied with the standard mechanism..

One rather strange feature of the interface seen through the Nokia is the invisibility of the title field. I see in the postings list that I've somehow managed to call it JU, but I really can't see how to change it. Maybe I'll change it later.

I seem to remember that the AJAX functionality used to work, but that's not the case since I upgraded the operating system. Maybe there's some setting that I need to update, but if so it eludes me. Anyway, this has taken quite long enough so I'll save the post as a draft and put the title right before I post it.

Browser Statistics (Hello, Robots)

During the current statistics trawl I started to wonder which browser technologies dominated among the visitors to the Holden Web site. November was an interesting month because both Microsoft and the Mozilla team released new products.

Here is typical information for the top ten client programs. Unlike previous statistics, Firefox users account for almost a half of the traffic to the site - usually Internet Explorer leads the pack with about 30%, but in November it was relegated to third position.



Internet Explorer 7.0 currently accounts for 2% of traffic, the first time it's appeared in the stats. The thing I found most surprising was the fact that more Firefox users were using the recently-released 2.0 version than anything else. 2% of users still use Firefox 1.0.

Rather less delightful is the fact that 20% of site traffic is spidering (though how come Yahoo has to crawl so much more aggressively than anybody else is beyond me). Perhaps the real message is that I need to put some more compelling content on the site!

A Tip o' the Hat to Ron Stephens

Looking at my web site statistics the other day I discovered, as you might expect, that the majority of referrals were coming from Google. However the notable second source is Ron Stephens' Awaretek site. Ron has been quietly generating Python reference materials and producing a Python podcast for years now, with vary little in the way of public acknowledgement. So thanks, Ron, keep up the good work!

Database API Tutorial at PyCon TX 2007

The list of PyCon tutorial offerings has now been finalised, and I'm happy to say that my Database API proposal has been accepted. Once again there is a great line-up of tutorials. It must have been hard for the program committee to choose this year's line-up since most of the offerings were strong and from well-respected speakers.

This doesn't guarantee that any tutorial will run: there have to be enough sign-ups for that to happen. This year there is also a tutorial on SQLAlchemy, and it looks as though the schedule will allow people wnating to understand how to use databases in Python to go through the DB API in the morning and then learn SQL Alchemy in the afternoon. I'll be conferring with Jonathan Ellis to make sure that I cover any necessary background for those who want to attend both.

I'm hoping that my tutorial will be well-received — last year's filled the room, and helped me fund my attendance at PyCon. It's always good to have guidance from students in advance telling me what their "hot-button" items are. If you're thinking of attending you can use this page to provide guidance on the content you'd like to see.

A False Sense of Security?

Although I am primarily a technical person I have broader interests than just that, including business and marketing. Sometimes these interests intersect in areas that affect a broader cross-section of the population. This is not a technical post.

One of my sources of occasional reading is Seth Godin, author of several books including All Marketers are Liars. He also runs a personal blog, and in the wake of the Thanksgiving rush he wrote this TSA-inspired post about the stupidity of the air transport security rules and the insensitive way in which they are administered. This really rang bells with me.

Godin courageously risks being thought a whiner by ridiculing the rules that allow a full three-ounce container of gel through the screening but disallow a five-ounce container with only one ounce in it. I've spent a lot of time in US airport security lines, and people aren't generally welcoming of critical remarks. Unfortunately many of the "security" activities that we have to go through are what Bruce Schneier calls security theatre.

At the same time as the TSA are forcing even the airline staff to submit to security screenings there are airports where construction staff are allowed airside simply by swiping an identification card. So the terrorists will be targeting the construction workers, not trying to smuggle six ounces of gel through the passenger channels.

Now I know I'm not the only person to suggest that the US government's initial response to the events of 9/11 was off-target. Benjamin Franklin was spot on in saying "Those who sacrifice freedom for security deserve neither". It seems that the majority of American citizens appear to believe that the government has duty to protect them from all risk from the cradle to the grave.

Said citizens, of course, are oblivious to the fact that it is impossible to eliminate risk completely: the aim of security is to raise the cost of a breach to a level that makes the breach uneconomic. But enough is enough. Dulles International has got to the stage where "going through security" can add over an hour to the passenger processing time, and I'm sure others are as bad if not worse. If things go on the way they are, people will stop flying because of the security measures.

What percentage of the American population use the air transport system? How much is being spent every day on airport security theatre, and how many lives is it saving? How much should we be prepared to spend to save a single life? How quickly could we have fresh drinking water available for all of humanity if we spent the money towards that goal instead? How many lives would that save? Which expenditure would make the USA more secure? Isn't it perhaps time for a reassessment of priorities?

Movie Moguls Need Control

In yet another example of movie mogul insanity the MPAA, in the guise of Paramount Pictures, is suing a small business that is ripping DVDs to iPods for customers who buy both at the same time. This highlights exactly why the Digital Millenium Copywrong Act was a bad idea, and it shows American politics up exactly at its worst: as a way for wealthy influence groups to get their own way at the expense of the buying public who have made them rich.

Don't even get me started on software patents.

Corporate governance always makes such a huge fuss about how corporations are really responsible to their stockholders. Who are the stockholders of the MPAA members, and why aren't they telling their managements to start respecting the customers who earn them their living?

Thunderbird QA Not Slipping!

Well, for my sins I have become a Thunderbird 3.0a1 user, and I can't say I'm happy. Several times the program has threatened to update itself, and eventually the promised (threatened?) upgrade arrived. [NOTE: apparently the 3-series builds are intended only for developers].

The error that originally caused me to update to the latest version is even worse in 3.0a1. Message contents are now being mangled on a regular basis. I have no idea whether they can be recovered correctly or not, and I am beginniing to lose confidence in what has until now been a reasonably reliable piece of software. It's ironic that I am suffering this problem even worse than I was because I wanted to help the project by making an accurate bug report.

If many other users are suffering the same problems I am then this issue needs to be addressed before it starts to give open source a bad name.

Thunderbird: Can't Help for Hurting

I've been an enthusiastic Thunderbird user for a fairly long time, and have only noticed odd quirks from time to time. One such quirk is the fact that occasionally a newsgroup article will show completely the wrong content: like the content of some other article altogether. This happened again today, and my first thought was to report the bug.

Now I've reported bugs in the past, and been told that they were duplicates of existing reports. So this time I decided I'd do the right thing and follow the reporting protocol. After all the guys who put Thunderbird together deserve some consideration, and I'm sure many users are cluelessly inconsiderate of the fact they are one amongst milliions and that they haven't paid a cent for some pretty high-quality software. I can't tell whether there is any description of such a bug in Bugzilla -- it might be bug 239665 or it might not.

It turns out that one if the things you are supposed to do is install the latest nightly download in a blank folder with a new profile and see if the same error occurs. This is not a prospect I relish as it will involve a certain amount of setup, but I do the download (nothing much else to do in my hotel room) and start the install. Unfortunately it turns out there's an error in the install, and I see a dialog box showing

Microsoft Visual C++ Runtime Library error R6034: An application has made an attempt to load the C rutime library incorrectly. Please contact the application's support teamfor more information.

Lucky me: I thought I had one problem, but now I have two. I feel duty bound to report this new error under the "Installer" category. Fortunately the installer completes, and I can run the new Thunderbird to confirm the same error occurs (though I stupidlu omit to bring my RSS feeds forward into my new version). So now I can go ahead and add to the bug report for the original problem, since unfortunately it hasn't gone away. My motivation is severely diminished, however, so this may not actually happen. At least I reported the installer problem.

One step forward, at least only one step back.

Stackless Python Continues to Amaze

I recently started tracking the Stackless Python mailing list. I've been interested in Stackless, the brainchild of the fearsomely clever Christian Tismer, for some time now. I recently acquired a client with active interests in that area, so it behooves me to stay up to date.

Stackless allows you to organise your work into tasklets (which as far as I understand it have replaced the original microthreads as the unit for scheduling: this is all pretty new to me, so apologies if it's incorrect). An interesting thing about tasklets is that you can pickle them, pass the pickle to another computer, unpickle them and resume the tasklet in the new environment.

My attention was grabbed by one recent thread in which a user explains that he was trying to perform such interactions across a heterogeneous environment with Power PC, SPARC and Intel-based CPUs and found that it worked between SPARC and PPC, but the Intel architecture appeared to be failing -- presumably due to the different endianness.

After a couple of messages surmising that this was hardly surprising, Christian came into the thread to explain that there was no architecture-dependent code in the pickled thread resumption functionality. Lo and behold, the original poster came back explaining:

I must apologize, it turns out my problem was due to the fact that I was using a different version of stackless on the intel machine. In case anyone is curious, resuming pickled tasklets across architectures is easy. Thank you for your replies!

Let me repeat that, just in case you missed it: resuming pickled tasklets across architectures is easy. As far as I can see this gives Python an amazing capability to produce applications with highly-distributed architectures. It's going to be interesting to see where Stackless goes with this, but it should be shouted from the rooftops. This is an advanced feature that represents a real advantage for (Stackless) Python in a world where everyone is wondering how they can accommodate the new multicore processors.

With Stackless, your CPUs don't even need to have the same architecture, but the feature will work just as well in the homogeneous environment that multi-processor computers provide.

What, Me, Unique?

Hardly, as the box below shows. But ..

HowManyOfMe.com

There are: 48 people with my name in the U.S.A. How many have your name?

... just the same none of those others have

• Fathered my son (hi, Simon!)
  
• Chaired three PyCons
• Written a book about Python
• Taught almost 3,500 students about networking and security topics
• Helped comp.lang.python readers to use Python

So there are lots of things that distinguish me from the doubtless equally worthy 47 others. It's always interesting to see the uses to which the web will put information. Twenty years ago HowManyOfMe.com would only have been imaginable as an online application. Now it just merits a raised eyebrow and a passing "good idea". Who knows what tomorrow's kids will take for granted assuming the preceding generations don't destroy the world.

Oh well, that's enough cheerful for today.

Thunderbird Politically Incorrect?

Someone just posted a rude but amusing remark about a regular comp.lang.python contributor, so I replied off-list with a snigger. Imagine my surprise when Thunderbird's spilling chucker failed to recognise the word and suggested a change to "nigger"!

Nice to be back on the blog. I've been working my ass off lately.

Moin Moin - Introduction for Developers (Thomas Waldmann, Alexander Schremmer)

MoinMoin was initially a one-man project, but the original author is less frequently available now and there is a core team that is collectively responsible for the code. The project has a large user base, which grew significantly when Unicode processing was introduced and enabled internationalisation.

For the rest of this presentation you should see the published slides, which unfortunately not only accurately represent the content of the talk but also encapsulate its presentation. I suspect (though I do not know them) that the speakers were perhaps nervous, and not therefore confident enough to talk ad lib in a foreign language to what must sometimes seem like an intimidating audience.

I found this a little frustrating since the few departures from reading the slides that the presenters allowed themselves made it clear that they both had an excellent grasp of a very popular Python web application. With more time at the conference I know I would have liked to talk to them both about possible applications and developments.

Literate Testing with Doctest (Marius Gedminas)

Marius started out by outlining the reasonable uses of doctest, both inline in programs and in files that are separate from the source code, and advised us not, for example, to put complicated setup or tests for obscure corner cases in the README. DocTest can be easily integrated with unittest, by defining a test suite that runs the doctests!

He then pointed out DocTestSuite and DocFileSuite for externalising the tests, and considered how to handle complicated setup. One way is to define setup and teardown functions, and you can also put code in test.globs, but these methods should be limited for ease of comprehension.

The advantage of doctest is in encouraging simlicity and interspersed narrative to explain the tests. Another option is to define a test module. The advice is to avoid mixing long tests and regular code in the same file.

In real life, of course, some features of doctest are difficult to use when data is not reliably repeatable (such as the addresses of objects in repr()-style representations). Also problematic are ellipsis-elided outputs, which are to be ignored, and ordering issues. Web testing is even more difficult, as responses are frequently state-dependent. marcus showed some examples of testbrowser, a simple system allowing web testing.

Doctest disadvantages include the disability to step through tests. Documentation (docstrings, README files) should be separated from unit and functional tests.

Questioning revealed that Zope uses a slightly different version of doctest from the Python 2.4 distribution, although "in theory" they offer the same functionality. Jim Fulton asked whether the speaker had looked at Fit (and apprently PyFit is available) but although Jim would like to see Fit and doctest integrated there was no experience with the packages in the room.

The only slightly problematic aspect of this presentation was a couple of times when the speaker made inaudible responses to inaudible questions from the chair. This would have been less troublesome if I had not been at the back of the room, or if the PA system had allowed audibility. Otherwise I got a lot of information in a very short time, well presented.

Working Together on the Web (Kevin Dangoor)

My whole EuroPython experience was a little disjointed, starting with arrival. I and one other delegate ended up on a bus that went, not to CERN but to a local railway station three or four kilometers away. The result was a late arrival, so by the time I took my place in this session Kevin was just starting to present his first example, how to adapt a simple application to the WSGI interface.

Using simple_app was the easiest, but Kevin also showed us how to do it by defining our own class. He then considered the problem of maintaining session state (though Kevin says he prefers not to use state).

WSGI is also helpful in such situations: its concept of middleware (which talks to the web server on one side and a web application on the other) can help. The example here was of a "Latinator" that translated into pig Latin. Kevin showed us how to define a class with a method that can be passed to the WSGI framework as the start_response argument. WSGI is good because there have been many contributions to the PyPI index.

Kevin next introduced us the the Paste system designed by Ian Bicking. This contains a confusing number of packages and modules, which makes it hard to describe and hard to climb the learning curve and start using it. He suggested this demonstrated that even in the Open Source world we are faced with the "build vs. buy" choice, even though "buy" is really an investment in learning someone else's package and build, as usual, is learning and understanding the base technologies you want to implement, and writing your own implementation.

In TurboGears Kevin made the choice to use as much existing technology as was practicable.

The fictitious web package author might decide to package up other dependencies, but this can cause problems for people with other versions of the components she chooses to bundle. This is the classic version dependency. The setuptools system has defined a cross-platform format for installation of Python packages, the .egg format. This can also be used to install packages with binary components, and is proving to be very usable.

The .egg format allows other systems to read the metadata about the package, which is useful in many environments where plugins are used, even in non-Python environments such as Eclipse. The discussion about how entry points were defined to plugins didn't seem terribly easy to understand to me - maybe I was just distracted by the extraneous materials, but I didn't really see the point.

Kevin closed by talking about Beaker, starting by showing us theat it depended on the MyghtyUtils package and defined two paste entry points. The examples at this stage started to leave me wishing I had attended the AJAX session instead, but this could just be me: others might have found the material relevant to their interests.

This presentation relied for contrast on rather too much use of "humourous" slides: a picture of an egg when talking about eggs, a picture of an RJ45 plug in a socket when talking about plugins. I have no objection to such devices, but they should either enhance understanding or be shown for a very short time to give everyone a quick smile before moving on.

Kevin knows his stuff and is always an engaging speaker. A tour de force, though, should really be allocated an hour, and I suspect this session should have covered more ground in less detail or vice versa.