Linux Journal Readers Choose Python Again

I've just noticed the latest Linux Journal, which contains details of their 2010 readers poll. In former years they attempted to distinguish between "programming languages" and "scripting languages" but last year and this they just allowed readers to vote for any languages they liked in either category.

The interesting outcome was that readers voted Python as both the best programming language and the best scripting language. Thanks, Linux Journal readers!

DjangoCon Retrospective

Tumultuous events seem to occur in clusters in my life, and the week of DjangoCon found me in the middle of one such cluster. This being more of a professional than a personal blog I concentrate here on the conference, where the tumult manifested itself first in the shape of a chance meeting with a congressman. The next day I helped the Conference Direct team prepare for evening registration and helped people sign in.

The three days of DjangoCon were a most stimulating time. The Django community has its share of characters, and being among so many was truly a joy. I ended up just up the corridor from James Bennett[1], who is also quite a social person, and so a certain amount of partying ensued (which I regret to say I was not really in the best mood to enjoy). James was happy to entertain the people I kicked out, and they were apparently happy to be entertained, so we achieved a pleasant win-win.

Conference time is usually fairly closely scheduled, and DjangoCon is no exception. I used my "keynote" to try to get people relaxed and in the mood to enjoy themselves a little during all the serious learning that was taking place [2]. Then it was into tracks and running. Lunch seemed to go down well and everyone appeared to enjoy the breaks. Due to the generosity of our sponsors it was possible to provide a continental breakfast this year as well as lunch and breaks. After James Bennett's keynote were the lightning talks, which I had to miss because I was running a Python programming competition (won by Justin Lilley, against some worthy competition). I finished the day with a quiet room-service dinner with Kirby Urner, who was acting as Snake Wrangler and my general factotum.

This reminds me to mention that the Django pony was again in evidence as was Naga, the PSF_Snake. Everyone who came to the conference received their own "My Little Pony" toy, and I believe I may have detected early signs that the Django community is moving towards being "ponied out". The pony was the subject of a question during my keynote that rather sandbagged me - I was unaware of strong feelings in certain elements of the community about the pony as a symbol. Having said which, honesty is the best policy and I said what I believe.

Day two moved on in strong style. I was much preoccupied with how best to try and accommodate a visiting speaker. In the end we just moved the whole afternoon forward by a quarter of an hour, leaving a 5:15 to 5:30 slot for Congressman David Wu. I don't really know what the audience made of being addressed by a congressman. The decision to invite him was a no-brainer for me. I would rather hear a bit of "why ever did you bring a congressman in to speak?" that any "why on earth didn't you ask the congressman to speak?" Congressmen aren't like buses, they don't come along on a regular schedule, and the particular congressman in question happens to have impeccable science and technology credentials. He certainly seemed reasonably well-briefed on open source, which is a big thing in Portland right now.

The early part of the evening was spent at a reception my company gave, which a couple of campaign staff also attended. I entertained them a little longer after the reception finished, and then went downstairs to grab a roomful of people to help me consume wine and food I had ordered for my room. This turned into a most stimulating party, but at midnight or so I realized I needed to get some sleep. Fortunately James sent a scouting party down, and a transfer to his rooms was affected with only one use of the verbal crowbar.

Later to bed than I had anticipated, I had to rise early to attend a breakfast meeting away from the conference. As I returned it was gratifying to learn that the bacon had been delivered for breakfast at DjangoCon. The hotel had to serve the savory pastries late and in insufficient quantity on day two because their bakery supplier let them down. They asked if we would like a per-delegate refund, but I suggested instead that they add bacon to the Thursday menu, which they duly did. Unfortunately the bacon was apparently set down in the middle of the general display, and although I made light of it at the time[3] it would have been nicer for the vegetarian population if it hadn't been thrust under their noses.

The day appeared to go well. At lunchtime I gave away books from APress and O'Reilly, our two media sponsors, and awarded prizes for various pieces of arcane knowledge ("What's the animal on the front of this book?"). The only slight logisitical glitch in the proceedings occurred right at the end, when the short length of the sprint preparations caught us on the hop. I got a call and immediately went down and "turned the bar on" so people could get a drink, and a couple of minutes later Nancy from Conference Direct arrived with the drinks tickets and it was business as usual.

I like to think that most people had a good time, and I have tried to deal with criticism as it was raised in so far as I could. Sometimes, though, people insist on waiting until they have left and then making negative remarks, and I have seen a few such:

What I learned at #djangocon: 1. Not enough core committers. That's it. Total bummer of a conference #circlejerk I hope this twitterer realises that the chairman and community volunteers are the right people to take this up with, and there has already been talk of an Exreme Django track.

Oh fast internet, how I missed you! Back from #djangocon.  I don't think the Internet service was that bad (considering that PyCon spends four times as much and still gets it wrong when they don't let Sean Reifschneider do it), and there weren't extensive complaints as far as I knew (except maybe one fairly short period). Or maybe the author is, like me, a happy consumer of fiber to the customer premises. You do get spoiled ...

What I disliked most about #Djangocon: I have a toy for a 3 year old kid as a shwag. The pony isn't funny anymore. And yet, such is the power of the six dollar pony that you kept it despite the fact that you hate it. Nobody was checking at the doors to make sure you took your pony home with you. You're just a schwag collector, aren't you?

@pydanny please be careful around those dirty Django men, @pythonchelle watch out too #djangocon. I don't remember who tweeted this, but I hope the dissatisfaction was momentary.  Dirty Django men? Not dirty (mostly), simply in way too large a majority. We need to work on that. And the dirt ...

http://isalexacoredeveloper.com/ is just cruel #djangocon I have to agree, it is, but I don't think that this is in any way DjangoCon's responsibility [4].

Overall I think most delegates enjoyed their conference. While running these events can be a little stressful it is rewarding to feel that the conference has helped in some way to advance the state of the Django art and allowed people some relaxation during the conference "white space".

My thanks to all DjangoCon US's sponsors, and to everyone from the Django community who helped to make the event such a success. I am already looking forward to 2011.

NOTE: DjangoCon US 2011 will be held at the Hilton Portland and Executive Tower on September 6-8.

[1]: OHWar quote: "Beer. My suite. Now."
[2]: As is perhaps indicated by the title: Matters of Little Consequence or The Care and Feeding of Ponies.
[3]: OHWar quote: "Screw the vegetarians."
[4]: OHWar quote: "Pinax is sort of a reusable application thing." As you will have seen if you followed the link, Alex's status has changed since the site was instigated, and the question is now answered in the affirmative.

PyCon Financial Assistance

PyCon US is again offering assistance to delegates who may not be able to cover their own travel and/or accommodation costs. Applications have now opened, and the announcement page contains a link to the application form (separately linked here in case that's all you need).

While there can be no guarantees of financial assistance I can think of many delegates who have enriched the conference by their presence only because they received assistance. And those from places where few people even travel abroad can attend and set minds on fire when they return home. So financial assistance is a real investment in broadening the Python community, and I hope it will continue to be so.

There's a deadline, though, so don't spend too long thinking about it!

Templating Systems

Recently Danny Greenfeld wrote a blog post called Stupid Template Languages in which he attempted to advocate the Django approach to templating. Basically this is to keep any logic in the templates as simple as possible (but no simpler). So it includes the ability to write for loops but not, for example, to define re-usable procedures with parameter passing.

Armin Ronacher reacted to this with a blog post of his own called Not So Stupid Template Languages, in which he attempts to defend Mako and similarly complex templating schemes. I don't see the Jinja2 examples he gives as countering the main thrust of the Django wish for simplicity in templating. He does, however, explicitly point out the weaknesses of the Django include mechanism, and highlights the "macro" feature of Jinja2, which is its equivalent of Django's include feature. Jinja2 macros require that data is provided to the call by argument passing rather than simply being extracted from an implied context. I agree that requiring the use of arguments is probably a good way to make the couplings explicit.

This doesn't however, negate the value of keeping complex logic out of the templates. This desire originally comes from Django's background in the newspaper industry, where the designers were definitely not programmers. So they didn't need a hugely complex templating languages, and were prepared to ask the programmers to generate in their views the values required by the templates. I have worked with tools like Mako, which includes the ability to write very complex logic which should really be part of the controller rather than the view. It doesn't help that Django's designers perhaps didn't understand MVC that well, otherwise they would have called views "controllers" and templates "views". Nowadays I believe they call it the "model view template" model, but that still doesn't avoid the ambiguity about views and whether they should include logic or not.

Even with a pure MVC architecture Jinja2's argument-passing enhancements would be an improvement - as Armin says, making the couplings explicit would be a Pythonic thing to do. The nice thing is that argument passing could be implemented as a backwards-compatible extension, where a call without arguments could fall back to using the context as a default global namespace.

Larry Ellison Prostitutes Java?

I just saw a tweet from a character called Zed Shaw. Zed is a colorful guy, and a man of determined opinions. If he thinks something is wrong he will say so, most likely dropping f-bombs freely in the process.

His first tweet was rapidly followed by another.

Thinking about it, I cannot remember installing any other language and have it dick around with the settings on some other application.

I thought possibly Zed had caught the toolbar from somewhere else (I came within an inch of simply throwing WinAmp away when it intruded on my browser's search facilities in a similar way), but sure enough there's a (sadly undated) article at java.com entitled How do I unistall the Yahoo! toolbar from Firefox?, a clear enough indication to me that Java is likely responsible for Zed's woes.The FAQ explains about not just the Yahoo! toolbar but also the Bing and Google toolbars, but it appears that only the first two are installed by Java.

I suppose I could be wrong, but it seems to me that this behavior is new since the Sun empire was assimilated into the hive-mind by the Oracle collective. The fact that you can deselect the install options is neither here nor there. The real question is, what's the strategy behind this change? I can hardly believe that any revenues this hare-brained scheme might bring in are of the slightest significance to the world's second largest software company. But why would the company that effectively owns Java (in so far as anyone can own a language) deliberately try to discredit it in this way?

We can only speculate. Let's make it interesting ...

Lazy Programmers, Installment 315

So, during a brief engagement with ING Direct's web site I receive the following error message:

 It was complaining because a) I had used a one-digit form for the month (i.e. 9/19/2010 rather than 09/19/2010), and b) because I had just entered the telephone numbers, as is my wont, as ten-digit strings.

What I want to know is, is there any user-centered QA in effect on this site? Who exactly decreed that dates had to be entered in that inflexible format? Who reviewed the stupid decision to require that specific format for telephone numbers, when it's actually much easier to simply throw away anything that isn't a digit, store that and then format numbers when they are printed? Finally, does anyone at ING care how much trouble they put their users to?

See, I've been in the business long enough to remember the days when computers were there to make things easier for user-type people, not for the programmers. This is clearly a lost approach, one sees so many ridiculous requirements such as the above.

What Do We Tell the Government?

This coming Thursday I am speaking at GOSCON, a conference organized with the specific intentions of informing governmental IT users about open source. Most of the speakers will be from the government side if the last such event I attended is typical, so I am giving a talk called No Free Lunch. The idea is to suggest ways that government can maximize the benefits of using open source, and provide open source teams with resources that will enable us to increase production, so this isn't just a talk for the Python Software Foundation - I want to represent a wider community if I can.

Twitter friends have already provided some useful ideas, but I will be happy to weave more in there if you can provide them. So, what would you like to see government (which in this context is principally the executive branch, but I am happy to include points for the legislative and judicial branches if you have them) doing for open source?

Pony Outlook Fine, Say Wu and Holden

There's been some reportage about the presence of Congressman David Wu at DjangoCon this year (to which the most I could do yesterday was briefly allude). After my Labor Day post I think I can safely say that Chairman Wu's presence was reassuring to the many at DjangoCon who might have been seriously inconvenienced by the pony shortage, which has been discussed in panels at the conference.

Of course it would have been terribly unfair to the Congressman to have asked public questions, and as he is in a busy campaign I know you would not have wanted me to delay him in your name. After a breakfast meeting this morning I am happy to be able to tell you, though, that his staff have convinced me there really is no sign of a pony shortage (and I doubt many reading this will have bought more ponies than I of late). In fact, had I known that I could have procured the ponies locally (Oregon price: 50c less than manufacturer's) I might have saved the cost of carriage, too. Next time I'll just call the store in advance.

If further reassurance is needed, I would like to note that after a third meeting with Congressman Wu I am happy to endorse him as a two-pony family man. Those ponies will be in safe hands, and the people of the First District of Oregon would be ably represented by a Congressman of such integrity.

Open Source and Politics Mix

Kirby Urner has written this great summary of the events of late Wednesday (DjangoCon time).

DjangoCon Triggers National Pony Shortage?

Hasbro Toys today took the unprecedented step of insisting that pony supplies were "absolutely not a concern" in response to persistent rumors that a foreign national had cornered the market  in ponies. A spokesman said "while it is true that there has been an unusual demand for ponies in the Portland area our normal stocking policies were quite adequate to meet demand. Anyone who wants a pony can have one".

The House Committee on Science and Technology is rumored to have directed its Subcommittee on Technology and Innovation to investigate this potential shortage of a strategically important resource in the creation of critical national web site infrastructure. The chairman of the Subcommittee, congressman David Wu of Oregon's first district, was not available for comment. Hasbro could not explain why American pony manufacturing facilities had opened on Labor Day, normally a national holiday.

Preparedness, Privilege and Discrimination

Coming very late to the party I noticed a blog entry from June about the JavaScript community's response to Google's financial support to allow more women to attend JSConf Europe. It sounds like it was indeed the usual real can of worms, and many of the comments show the usual lack of appreciation of the problems from the privileged side of the issue (in this case men, since it is a gender issue and women appear to be about as well represented in the JavaScript community as they are in the Python world, which is to say hardly at all).

The reason I found it interesting was that this year, for the first time, PyCon also used Google's support to fund the attendance for more women, who came from as far afield as Roumania and India. I was gratified to be told by several of those who would otherwise not have been at the conference how happy they were to have had the chance to attend. (In truth I had little to do with it: Google should get the credit for the funding, and the actual hard work was done by Peter Kropf and Gloria Willedsen). In the Python community's case there was one short period of adverse comment on the #python IRC channel, which resulted in my exchanging emails with one person to explain why I thought it was a good idea to encourage more women to be at PyCon. End of story, except that in 2010 women represented 11% of PyCon attendance, up from 2% the previous year. I count that as one of the better results of my time as chairman.

Now I am not saying this to be smug, but because I believe there was a reason for the difference in the reactions. Last year the PSF, at Guido van Rossum's urging, started a diversity mailing list which discussed the questions of race, gender and other discrimination extensively and sometimes acrimoniously. Eventually this led to a proposal for a "diversity statement", which was referred to the membership where it triggered another round of extensive and sometimes acrimonious discussions, leading to a referral back to the diversity list and a further proposal which was finally accepted by the membership more or less unchanged and adopted by the Board:

The Python Software Foundation and the global Python community welcome and encourage participation by everyone. Our community is based on mutual respect, tolerance, and encouragement, and we are working to help each other live up to these principles. We want our community to be more diverse: whoever you are, and whatever your background, we welcome you.

This may not be the best statement ever, but if anyone bothers to look it does make it clear that these issues have been addressed. Thus anyone who feels discriminated against can decide that at least they would have a chance of a fair hearing should they choose to complain (which, sadly, I imagine most don't, instead choosing to vote with their feet). Similarly, anyone about to indulge in discriminatory behavior might think twice before doing so.

The hidden benefit of this long-drawn-out process was the creation, on the diversity list, of a corpus of varied individuals who had discussed these  issues and hammered out a shared approach to the problems that included a refusal to punish individuals for things done out of ignorance. It also meant that when one speaker used a slightly ill-advised graphic in a presentation the issue was dealt with then and there in a very direct manner without any recriminations, and I didn't even get to hear about it until  much later that day. The speaker was advised that the material was inappropriate and that therefore the slides and the video of the talk wouldn't be published, and hopefully left without feeling that they weren't welcome at next year's conference.

I hope that the JavaScript community manages to develop its own understanding of diversity issues and its own process for dealing with them. I know it took up a lot of my time as PSF chairman and gave me some uncomfortable moments (and does not exempt me from the results of my own stupidity in the future), but I am glad it led to a tolerant community process that nevertheless has made it clear that discrimination is not acceptable.

The Hiring Smiley Curve

I recently commented on how hi-tech companies seem to advertise for "rock star" developers. This appears to me to be a little shortsighted. Maybe Google have enough money to do it, but the average company looking for a singer wouldn't be able to go out and hire Mick Jagger or Van Morrison (yes, I know, I am showing my age - it will become obvious this isn't an accident). So why do they think they can afford the software equivalent?

As I mature (that sounds so much less pejorative than "grow older", doesn't it?) I have realized that my decision to eschew the corporate world and plough my own furrow wasn't entirely disastrous. I have been self-employed, or an employee of a company which I owned, for over twenty years now. The last large company I worked for was Sun Microsystems, and I left in 1988 after realizing that I was a misfit for the corporate environment. Since then I can honestly say I have never worked for a more charming or delightful person. Nowadays my boss never hesitates to take my interests into account.

Just occasionally, I have toyed with the idea of third-party employment, most recently with Google (twice). The first time I explained before going for interview that I was not prepared to relocate to the West coast . At the time I was moving back from the UK to the USA, and had just bought a house in Virginia, so I was a little surprised to learn three weeks after my interview that "we aren't prepared to make a remote hire, but would consider you for Mountain View". Large company mentality: ignore what the potential recruit wants, and try to hire them anyway. No sale. About a month ago they called me to ask if I would consider employment in the DC area, but it turned out they were only hiring for Java projects. I used to write Java but I'm all right now, so the recruiter and I agreed to part company amicably after ten minutes on the 'phone.

None of this amounts to a hill of beans but I was encouraged about a year ago, when I was talking to someone in New York City about experience levels and consulting opportunities, to learn about the "smiley curve" theory of work rewards. In brief, this theory says that you hire young raw recruits because they are full of energy and don't cost much. As people become more experienced they expect more money but their productivity doesn't go up commensurately, so they are less desirable but you need them because there aren't enough young turks to do all the work. Then, as you mature, your desirability goes up again because (direct quote, as far as I can remember) "you have seen everything and you know everything".

If true, this is quite encouraging. There must be lots of companies looking for someone as experienced as me. The only question now is whether they can afford me!

Apple Going Over the Top?

Yet again I rejoice that I am not an iPhone user. Indeed, given this latest news I might well just chuck  my Mac mini away in protest (no, you probably don't want it - it's an aging obsolete PPC mini from about six years ago).

In news from the Electronic Frontier Foundation I learned today that Apple has applied for patents on method of spying intrusively on the users of their devices. Here's a partial list of possible applications:

• The system can take a picture of the user's face, "without a flash, any noise, or any indication that a picture is being taken to prevent the current user from knowing he is being photographed";
• The system can record the user's voice, whether or not a phone call is even being made;
• The system can determine the user's unique individual heartbeat "signature";
• To determine if the device has been hacked, the device can watch for "a sudden increase in memory usage of the electronic device";
• The user's "Internet activity can be monitored or any communication packets that are served to the electronic device can be recorded"; and
• The device can take a photograph of the surrounding location to determine where it is being used.

So enjoy your iPhones. I won;t be doing business with a company that thinks this way. Sigh. I suppose that means the new laptop will have to be PC based.

Windows Vista Mystery Shares

For reasons best known to Microsoft, when I try to delete a folder which has been shared (through the Explorer interface) it takes forever to complete. This would not be so bad if there were just one or two shares, but sadly (for reasons best known to Microsoft) a large number of folders randomly appear to have become shares (see the screen dump of a portion of my home directory at the right). I have no idea how these folders became shared. It certainly wasn't any intentional act of mine, and heaven alone knows what this does to performance.

Now, you are probably wondering why I don't just switch off sharing before I delete the directory. The answer to that is that although Windows is displaying the folders as shared, it doesn't really seem to believe that they are shared. So there doesn't appear to be an easy way to switch this sharing off.

If I had some idea how it had been switched on in the first place that might help, but as with so many other aspects of Windows performance this remains a mystery. If someone cold offer some insight I'd be happy to find out what's going on here.

Schmidt Foot-In-Mouth Attack Continues

Wow, two consecutive Eric Schmidt posts. But this really is quite newsworthy. An interview with Wall Street Journal published on Saturday claims

He predicts, apparently seriously, that every young person one day will be entitled automatically to change his or her name on reaching adulthood in order to disown youthful hijinks stored on their friends' social media sites. 

This really is the most patent hogwash. Surely someone like Schmidt, with a brain the size of a planet, could foresee that if such name changing were to become commonplace it would inevitably lead to the creation of services that mapped between past and present-day identities? Given the ability to identify images recently demonstrated by sites like tineye.com it's only a matter of time before changing your name will no longer be a way to erase the records of your misdeeds.

This makes it all the more important that privacy and basic information security become high school subjects, but alas the corporate overlords that have the ear of government in most developed (and many less-developed) countries will be attempting to make sure that isn't a priority, because it isn't in their interests.

Eric Schmidt Looks Forward to Big Brother

"You only need to give up
just a little bit of freedom"

Well, Google's senior management are really keeping us guessing this month. I am not really sure any longer whether the company actually has any coherent point of view on individual privacy. Nowadays it seems you can't even expect consistency. In a startling repetition of his prior assertions that not only is Internet anonymity dead but that he wants to dance on its grave. According to a Read Write Web report Eric Schmidt, CEO of Google, included the following comments in his remarks to the Techonomy conference:

"The only way to manage this is true transparency and no anonymity. In a world of asynchronous threats, it is too dangerous for there not to be some way to identify you. We need a [verified] name service for people. Governments will demand it."

Any government that requires individuals to give up their rights to anonymity is a government past its sell-by date. To my mind this reveals further insight into Schmidt's "what's good for business is good for the people"  view exemplified by the much-discussed recent joint statement with Verizon on network neutrality. It's fairly obvious that Schmidt sees government's role as paving the way for corporations to increase their profits, not preserving the freedoms of the citizenry who elected it. This is so far from "government of the people, for the people and by the people" that it's apparently time "do no evil" was replaced by "make more money".

"But that would hurt Google's
stock price!"

Yet it was only in May (yes, three months ago) that Schmidt was publicly suggesting that as far as Google was concerned "privacy is paramount". Does he really know what the priorities are any longer? It seems like we can dismiss any further utterances as the self-serving flip-floppery of the 129th richest man in the world. What does he really think? Apparently it depends on which way the financial wind is blowing. The really depressing thing is it's transparently obvious that here we have a man who will do well in politics. "The Best Democracy Money Can Buy" indeed.

Tests That Test Your Tests

I just had an interesting experience with Steve Miller, the technical editor of the Python classes I am writing for O'Reilly School of Technology. We are just getting to the end of the second of four courses, and I like to think that we are moving right along: in the final lesson students write a simple GUI-based program that searches for and displays e-mail messages stored in a MySQL database.

I introduced test-driven development in the second course. Not only does this encourage good habits in the students, it also makes it somewhat easier to test some of their exercises (though I still do not have a good approach to testing Tkinter-based GUI applications). In time-honored fashion we start with tests and a program full of stubs, and then expand the stubs to pass the tests. For the email database the initial API is very simple: there is one function to store messages and two others to retrieve them, by primary key and Message-Id.

In the final chapter I start out with a very simple database table that stores the body of the message as a LONGTEXT column. The only other columns in the table (to start with—it gets more complex later) are the automatically-generated primary key and the Message-Id Header. The tests use a setUp() method that completely re-creates the database table and populates it from messages stored in a bunch of files:

 FILESPEC = "V:/Python2/Lesson12/MailData/*.eml"  
 class testRealEmail_traffic(unittest.TestCase):  
   def setUp(self):  
     """  
     Reads an arbitrary number of mail messages and  
     stores them in a brand new messages table.  
     DANGER: Any existing message table WILL be lost.  
     """  
     curs.execute("DROP TABLE IF EXISTS message")  
     conn.commit()  
     curs.execute(TBLDEF)  
     conn.commit()  
     files = glob(FILESPEC)  
     self.msgids = {} # Keyed by message_id  
     self.message_ids = {} # keyed by id  
     for f in files:  
       ff = open(f)  
       text = ff.read()  
       msg = message_from_string(text)  
       id = self.msgids[msg['message-id']] = maildb.store(msg)  
       self.message_ids[id] = msg['message-id']  

There were two relatively simple tests, one to test each of the retrieval functions in a fairly simplistic way by verifying the correspondence between the primary key values and the Message-Id headers using the msgids and message_ids dicts created during the set-up. The initial stub under test only implemented the store() function, so these tests were initially expected to fail:

   def test_message_ids(self):  
     """  
     Verify that items retrieved by id have the correct Message-ID.  
     """   
     for message_id in self.msgids.keys():  
       pk, msg = maildb.msg_by_id(self.msgids[message_id])   
       self.assertEqual(msg['message-id'], message_id)  
   def test_ids(self):  
     """  
     Verify that items retrieved by message_id have the correct Message-ID.  
     """  
     for id in self.message_ids.keys():  
       pk, msg = maildb.msg_by_message_id(self.message_ids[id])  
       self.assertEqual(msg['message-id'], self.message_ids[id])  

Steve and I had both run this code under much the same conditions as the students would, and verified that the tests did indeed fail due to the AttributeError exceptions raised by the missing msg_by_id() and msg_by_message_id() functions. Later steps have the student implement these functions, which makes the tests pass.

We were somewhat surprised to find when Steve ran his final checks that the tests were now passing, even though he had reverted to the original module with no implementations of the message retrieval functions! It took me the best part of an hour chatting with Steve to eliminate everything I could think of that might be wrong: no .pyc files left lying around, no odd path settings that allowed import from other copies of the code, and so on.

We finally tracked the issue down to something stupidly simple, as is often the case with bugs that have you scratching your head for an extended period: for production purposes the data files had been moved to an area where all students could share them (each student has their own V: directory). This meant that the setUp() method was not inserting any rows into the newly-created message table. The empty table in turn meant that neither test_message_ids() nor test_ids() was running the the body of the for loop, and consequently no AttributeError exceptions were being raised. The tests were passing even though the functions they were supposed to test had not been implemented!

My solution to this was to add a further test to verify that the table was not empty. That way, even if the other tests passed, this one would fail:

   def test_not_empty(self):  
     """  
     Verify that the setUp method actually created some messages.  
     If it finds no files there will be no messages in the table,  
     the loop bodies in the other tests will never run, and potential  
     errors will never be discovered.  
     """  
     curs.execute("SELECT COUNT(*) FROM message")  
     messagect = curs.fetchone()[0]  
     self.assertGreater(messagect, 0, "Database message table is empty")  

The check could have been added to one of the other tests but it seemed to make more sense to keep it separate, since several tests relied on the table having been populated.

In this case the error condition was that the test were passing! I am happy about this because it clearly demonstrates the value of test-driven development, even though the result I was getting was normally the desired goal of testing. It has also taught me to be more careful about tests in loops: if there is no guarantee that the loop body will execute then the tests inside it can be completely useless.

DjangoCon US is International

As registrations for DjangoCon US grow it's been interesting to see where people are coming from. I originally thought that it would be US-only. While US delegates dominate the lists as you might expect, we have people coming from all over the world. Here's a graphic showing the distribution of delegates across the globe.

I think it's a measure of Django's excellence that DjangoCon attracts people from so far away. I well remember when Jacob Kaplan-Moss and Adrian Holovaty first came to PyCon in Washington DC to describe the system they were putting together. At that stage Django wasn't open source, and the encouragement of the enthusiastic PyCon audience was a major factor in its becoming so. The software has come a huge distance in a relatively short time, and is now a major Python success story.

There are still places left at the conference if you would like to register. Portland is an intriguing city that offers a warm welcome to visitors, and the Doubletree is a green hotel with excellent accommodation. The conference room rate is only guaranteed until August 13, so make sure to book your accommodation soon.

Wave Goodbye

So Google's blog announced today that development of Google Wave "as a standalone product" will end because "Wave has not seen the adoption we would like". It's kind of a shame, because Wave was intriguing, but as an infrequent Wave user I found several issues that made it less than user-friendly. So here are a few points that developers might like to take home from the train-wreck that is Wave (100 developers for two years is a substantial investment, even for Google). The servers will continue to be available "at least until the end of the year".

1. Don't try to replace standard GUI components with inferior and non-intuitive substitutes. The Wave scrollbar was a user interface disaster, and a source of frustration to many of the users I interacted with.
2. Don't promote technologies that depend heavily on high-bandwidth connectivity, or at least not for Internet use. Many times I was left frustrated, not knowing whether the Wave had crashed or whether it was simply waiting for a server response.
3. Realize that even the best technologies need marketing and publicity. 80%+ of desktop computer users don't use Windows because it's the best system, they use it because it's the best alternative they know about. If people don't know about your technology they won't use it, and techies alone probably aren't the right user base to make a product viral.

In some ways I am sorry that the Wave didn't succeed as so many techies apparently thought it would. In other ways I am disappointed that the technology didn't really deliver on its promise. It's one of those things that needs to be ubiquitous to succeed.

Google's misstep with Buzz earlier this year probably didn't help either - it led to distrust about Google's intentions with regard to (or, worse, competence at securing) users' personal data.

So whatever the next big thing on the Web is going to be, it isn't going to be Google Wave. RIP.

Paypal Pending Balances

According to the PayPal user agreement:

10.7 Reserves. If you receive Purchase Payments, PayPal, in its sole discretion, may place a Reserve on funds held in your Premier or Business Account when PayPal believes there may be a high level of risk associated with your Account. If PayPal places a Reserve on funds in your Account, they will be shown as “pending” in your PayPal Balance. If your Account is subject to a Reserve, PayPal will provide you with notice specifying the terms of the Reserve. The terms may require that a certain percentage of the amounts received into your Account are held for a certain period of time, or that a certain amount of money is held in reserve, or anything else that PayPal determines is necessary to protect against the risk associated with your Account. PayPal may change the terms of the Reserve at any time by providing you with notice of the new terms. 

 What this doesn't say is that PayPal will also deny you the ability to earn interest on those funds while they are held in reserve.  Because Holden Web is organizing DjangoCon we currently have a substantial balance (which will soon disappear once the outgoing starts). So PayPal have decided that they can place $20,000 on reserve, thereby refusing me the right to earn interest on that money.

Naturally the "notice" I received just said that the funds were being held on reserve until further notice, and when I inquired about this the "explanation" I received didn't explain anything at all: it simply pointed me to the user agreement and suggested that I could find out what information they had used in the decision-making process by taking out a subpoena. Definitely not cool. In terms of customer service PayPal are scoring about 1 out of 10 here.