Natural Language Toolkit

I have just seen an announcement for the latest version of the Natural Language Toolkit. This is an excellent example of a useful open source software project and, as the announcement says:

It comes with 50k lines of code, 300Mb of datasets, and a 360 page book which teaches both Python and Natural Language Processing. NLTK has been adopted in at least 40 university courses. NLTK is hosted on sourceforge, and is ranked in the top 200 projects.

With this kind of software available for download, there will be many more unsuspecting Python users. The really nice thing, of course, is that the users aren't interested in using a particular language, they are interested in solving problems in a specific domain. The fact that Python lets them do this is a testament to its usefulness.

Goodbye Site Meter

Just goes to show, you can't be too careful. A while ago, wanting to know a little more about the traffic this blog sees, I added a widget from Site Meter to the layout.

Today I discovered that Site Meter have done a deal with a third party company, and that the code they send out includes references to specificclick.net pages, allowing tracking via cookies. So I have removed the widget and am looking for a new site metering technology.

It's not a huge security problem, but it means that you can be tracked across multiple sites that each use SiteMeter's logging widgets. This is allowed by their terms and conditions:

Site Meter may from time to time also authorize and facilitate the use of cookies from trusted third party business partners to gather and aggregate additional, anonymous, and non-Personal Information data from general internet visitors for the purpose of providing our customers with additional information about their viewing audience.

It's true that as long as SiteMeter don't release users' personal information there's nothing that specificclick.net can do over and above correlation of visits by anonymous users across multiple sites. But if one of those sites collects personal data and chooses to release it, that privacy is over. I decided I'd rather not subject my readers to that risk. If you have any such cookies in your browser I'd recommend you delete them. If you don't know how to do that you can find instructions at this About Cookies page.

Does python.org Need a Better Navigation Bar?

Someone recently posted on c.l.py that the python.org site didn't make it obvious where to post a bug. So I snapped the navigation bar from the home page (though of course the navigation bar isn't necessarily where the feature should be added). How do readers think it could be improved?

[Note added later: clipmark feature was unsuccessful at capture, so I have replaced it with a PNG capture to better show the site]

Microsoft Strategy is Patently Ridiculous

A recent Fortune article, Microsoft claims software like Linux violates its patents, suggests that the Ballmer empire is about to start seeking royalties from users of open source software whihc, the company claims, violates 235 of their patents.

I don't think they have thought this through. The US Supreme Court has so far issued no ruling on whether software is even patentable, despite the Patent Office's ridiculous willingness to issue patents on techniques that fail even the simplest test of obviousness. When the most powerful software company in the world starts throwing its weight around to gain revenue from those patents it will force the issue somewhat.

The inevitable result will be a Supreme Court ruling that inevitably weakens, or even removes altogether, the protection that patents have been assumed to provide by those who have invested heavily in them. Microsoft senior VP Brad Smith claims, for example, that the Linux kernel violates 42 Microsoft patents.ourse the joke is that nobody has any idea how many patents Microsoft products violate because, unlike the open source projects Microsoft complains about, the code that comprises them isn't available for public scrutiny.

Python Slithers into Systems

Nice to see ITA, a PyCon sponsor, getting publicity for themselves and Python in eWeek: Python Slithers into Systems. ITA aren't exclusively programming in Python, I happen to know they use Lisp and at least one compiled language as well. This demonstrates yet again that Python is a pragmatist's language, and it's definitely rising in visibility.

Another Great Python Blog Entry

I've been following Doug Hellman's Python Module of the Week series, but I already know most of the modules he'd covered until along came PyMOTW: logging which describes a module I have always found difficult to understand in terms that make it comprehensible. Nice job!

Wyatt Baldwin Blog - Google Maps

Wyatt Baldwin has recently made a couple of interesting blog entries (I'm just catching up after an extended Windows repair session [spit, spit]. In Google Maps Encoded Polylines he uses Python to draw shapes on Google Maps, and in Creating a (Google Maps) Tosca Widget he explains in considerable detail just how to do that. Both great posts demonstrating Python's power, and much kudos to Wyatt for this excellent work.

Debian -- python-samba

I wonder what's happening with python-samba. I came across the Debian package while looking at a Samba bug notice, but so far I haven't heard about usage. The package description does say

At the moment their status is "experimental" but they have been reported to work well.

Rethinking the Linux Distribution

Just one or two quotes from this interesting ONLamp.com piece.

Many well-known Linux distributions already use Python in their key tools. Red Hat's Anaconda installer, and Gentoo's Portage package manager are two examples. Ubuntu (the top distribution for the last 12 months, according to DistroWatch) "... prefers the community to contribute work in Python."

and

"Among the high level languages, Python seemed to be the best choice, since we already use it in many places like package build scripts, package manager, control panel modules, and installer program YALI. Python has small and has clean source codes. Standard library is full of useful modules. Learning curve is easy, most of the developers in our team picked up the language in a few days without prior experience."

So why aren't you using Python? (Don't tell me, you are ...)

Writerly Advice

Kurt Vonnegut died recently, but one of his legacies is this advice for writers. We could all read it and benefit from it, but I hope it will be of particular use to anyone responding to my PSF blog entry encouraging people to write Python articles for Sys Admin magazine.

Open Source "Increasingly Used for Critical Applications"

The market research company Forrester has just issued a report saying (among many other things) that more than 75% of respondents "agreed that open source software was making an important or very important contribution to improving efficiency and concolidating IT infrastructure". Other highlights include the fact that concerns about intellectual property are fading (presumably as SCO's cases are seen more and more clearly to be the FUD of a desperate last-ditch bid for survival).

The closing advice for would-be adopters comes in two major chunks:

1. Lower Internal barriers to Open Source Adoption - clearly it's time for policies to be refined to give open source a level playing field against proprietary products; some organizations still have blanket bans on the evaluation, let alone use, of open source.

2. Identify Services Before You Commit - many corporate users appear to be concerned that appropriate support services don't exist for open source products. While this isn't universally true, the open source communities equally need to acknowledge that corporate users might not feel completely comfortable relying solely on volunteer newsgroups for support.

It would be good to start building international federations that can collectively offer 24/7 support for open source projects, with specified service levels. That's going to be a challenge for the open sourcerers, but if they can solve it then they might even manage to get on the gravy train before it leaves the station. While this may not be everyone's dream it would be nice to channel funds in directions that allow (or even encourage) the development more and better open source software.

DRM Saga Continues: No Surprises

Well, the content protection debacle grinds on. The users continue to be unimpressed with digital rights management, and the hackers and crackers continue to break the harebrained industry schemes with monotonous regularity. Digg temporarily tried to censor material as a result of a blanket cease-and-desist threat, got it in the neck from their users, and apparently learned their lesson.

Favorite quotes from the latest discussion:

As Joe Rogan's character on Newsradio once quite accurately quipped, "Dude, you can't take something off the Internet.. that's like trying to take pee out of a swimming pool." The content providers have attempted to do exactly that, remove pee from the proverbial swimming pool that is the Internet and, as we've witnessed so many times before, they've failed miserably. [1]

and

There isn't a single known DRM system worth cracking that hasn't been cracked, multiple times; AACS will likely be no different. [2]

When will the recording and film industries learn that the "lost income" from content pillaging by dishonest consumers isn't "lost" at all? It's what the retail trade calls "wastage" (some things arrive spoiled, some get spoiled, some are stolen). Their revenues are what their revenues are, and the overall level of dishonesty is what it is. The prospect of additional billions is illusory, because people who don't pay for content will simply stop consuming it if they can't get it for nothing.

So the joke is that if the industry ever achieved their total-protection nirvana they will have shot themselves in the foot, because even bootleg copies are positive marketing. Sometimes greed is so sad. Were I a stockholder in RIAA member companies I would be furious at the waste of effort. I wonder if the senior staff of studios actually pay for the DVDs they take home. If not, aren't they too stealing from their stockholders?

I am Smalltalk

Well, apparently I am Smalltalk. According to this highly reputable quiz I found on the Internet :-)


Which Programming Language are You?

Given that I was responsible for an early UK port of Smalltalk-80 (the heavy lifting being done by Mario Wolczko) I don't particularly mind this verdict, but I don't do Smalltalk any more. It's wrong about the objectification, though.

Other People's Time

Would you rather work with a contractor who gives you a due date and sticks to it or one you could never be entirely certain of? If the project slips, how good will communications be?

Seth Godin here relates why it's important to keep public events to a schedule. Conferences aren't about the muckety-mucks, they're about the hard-working stiffs who've forked over their cash. Well, the conferences I organize are, anyway.

If a meeting's not about the participants, why are they there?

Python Acknowledged at IEEE

The May/June edition of the prestigious Computing in Science and Engineering (CiSE) magazine is a single-subject issue on the topic of Python. This is surely a recognition of the language's value to scientists, and a vindication of the work put in by scientists all over the world to apply Python to scientific tasks.

CiSE is a joint publication of the IEEE Computer Society and the American Institute of Physics. Unfortunately not all articles are available electronically to non-members.

Bet Your Business on Python

It is encouraging to see that increasingly businesses are depending on Python to give them a strategic advantage in their marketplaces. The most recent comment of note comes from yet another PyCon sponsor, ITA Software, who use Twisted quite heavily in airline scheduling applications. They are a new phenomenon to the airline business: a company that provides responsive service without the use of mainframe heavy-iron. You can read the whole eWeek article to get more context about ITA's use of Python, but the quote I liked best was from Dan Kelley, ITA's director of application integration:

"So we think, absolutely, that it's ready for prime time. What we're doing is saying to a billion-dollar business, 'Yes, we can write components in this particular programming language, and they will keep your airline running.'"

Note that ITA don't use Python exclusively, and they see one of the advantages of distributed systems as being able to choose appropriate languages for each component.. I don't believe anyone thinks Python is perfect - it's a waste of time to seek perfection in a programming language because the concept is so subjective. Just the same it's nice to see real businesses coming out with this kind of informed opinion about Python in fairly mainstream publications. Quite apart from the general advocacy benefits it is useful, among other things, as ammunition against the dynamic language bigots should any such ammunition be needed.

MSDN Webmasters: Read Peter Norvig!

Peter Norvig's an interesting chap. I mean, head of research for Google, that's got to be an interesting job, right? One of the blogs in Planet Python (I can't remember whose, unfortunately, because the "How'd I get here" button in Firefox can't tell me what Thunderbird was looking at when I clicked a link (oh, all right, three minutes research tells me it was Peter Bengtsson)) recently pointed me at a recent Norvig gem, the spelling corrector in 21 lines of Python.

I don't believe I've previously mentioned in these pages that I am dabbling with ASP.NET. Shrieks of horror resound around the halls of Holden Web. It's been interesting in the sense of the Chinese curse that dooms you to live in "interesting times". The learning curve is steep (and any regular readers of comp.lang.python will know that that metaphor has been lately subjected to the Belgian logic chopper, to nobody's very great illumination). But I digress.

One of the things I have spent a lot of time doing, given the large-ish nature of the framework I am trying to learn, is look things up in the MSDN documentation. Anyone who has corresponded with me will know that the standard of my typing is roughly the same as if you were to pay a squirrel to dance on the keyboard, which is to say I make typographical errors with a regularity that makes monotony seem interesting - the Secret Service characterize my typing style as "frequent use of backspace key". Consequently I have, in the last 24 hours, entered "sqldatasourec reference" into the MSDN search page no less than three separate times (see, when I find a way to bollix a word up I stick with it - I am only half a touch typist, so short entries not for human consumption are even more frequently wrong).

Three times I have been confronted with the helpful message "We're sorry, we couldn't find any results containing sqldatasourec reference". Now this is exceedingly strange. In trying to verify this behavior I made yet another error, entering the search string as "sqldatasource referenec", and while I saw the same(-ish) error message there was above it the helpful, dare I say Googlesque, question "Did you mean: sqldatasource reference" - which, of course, I did. Perhaps the correction corpus should be extended to include the subject material?

Just for devilment I entered the same two search phrases into Google's engine and sure enough it corrected them both. The obvious conclusion must be that Google know more about the content of MSDN than Microsoft do, but surely it wouldn't be fair to say that. Or maybe it would ...

Help Wanted!

Another post from Doug Napoleone says (among many other things) "I need to take the very clean and and elegant code of Steve Holden and stuff in into the somewhat organic and undocumented PyCon-Tech code."

It's strange how other people's perceptions of our code can be so different from our own. I am conscious of several areas where that code (published, by the way, as a part of my PyCon 2007 tutorial) really needs revision. But anyway, thanks for the tip of the hat, Doug.

Anyone wanting to help to improve the software support for PyCon should sign up for the PyCon Tech project. It would be a great way to get started in Django in a supportive environment.

Python is Not a Religion

The title of this piece is a quote from a post by Doug Napoleone on the subject of Python advocacy. Another quote: "Advocating Python for the sole purpose of promotion is idiocy." I quite agree with that one too. I don't really understand quite why one of my blog entries is referenced as ironically hypocritical, but I suppose I can live with that.

Nor am I sure who Doug is referring to as those who feel there is no need for Python advocacy. I certainly can agree, though, that there is absolutely no need for religious wars when it comes to choice of programming language, and that the appropriate question to ask is "is Python a good way to solve my problems". Anyone who sees Python as the solution to all programming problems is perceiving the language as a hammer and the problems as nails. There are many things that Python is good at (and many more it could be good at if someone wrote the right applications), but it's no more a panacea than any other language.

Doug's blog post appears to be a call for enthusiasts ("passionate people" is what he calls them) to come to the aid of the language. The sad fact is that the majority of the users of any technology (or any religion, come to that) - even users who profit vastly from its adoption - much prefer to be passive consumers than advocates, and this is the way of the world. Passion is all very well, but ultimately it needs to be directed to a goal, and shouldn't the goal of Python advocacy be helping people to solve technical problems more effectively?

While passionate people will help, I think advocacy needs planning as well as passion, and it's only when the two go together that language advocacy really helps. Support for the existing community is important, but so is increasing awareness of Python outside that community. I'm not convinced that most community members have the stomach for that task. It remains to be seen whether Python users are the best advocates for the language, but rightly or wrongly the Advocacy Coordinator position is currently planned as follows:

• User groups how-to and content resources (40%)
• Web site content showcasing Python capabilities (40%)
• User group infrasructure support (5%)
• Reactive and proactive response to queries and support requests (15%)

I am sure that the Coordinator, Jeff Rush, would love to have passionate people knocking on his door and asking how they can help him. Then he would have more time for other efforts that the PSF has asked him to back-pedal on for the moment. Ask not what Python can do for you ...

Buildbot Machines Wanted

I know there are quite a few hosting companies now who represent themselves as Python-friendly. Rather than trawl through the Wiki pages, however, and email each one separately I am sending this request to the comp.lang.python list (and putting it on my blog) in the hope of attracting those companies who are more involved with the Python community.

There is a particular problem, highlighted recently by this comment from Dennis Lee Beiber:

Too many 3rd-party modules still aren't available in 2.5 versions for my tastes...

This applies particularly (though not exclusively) to the Windows platform, for various reasons -- the most common one is that Linux developers frequently don't have a Windows machine available to help them test their builds and ensure that distributions are available.

I am trying to address this problem, initially by making hosted Windows machines available for use as buildbots. I already have agreement from Grig Gheorghiu (who maintains the PSF's buildbots) to try and support these efforts, and from Microsoft to consider providing appropriate software.

What I don't have is hosting companies offering me space on machines in their racks. If anyone reading this can help out I'd appreciate it if they would get in touch with me (replying to the newsgroup post or commenting on the blog entry should do it). These machines would need remote desktop access so they could be managed without physical presence.

The intention is to try and shorten the "version lag" so that new versions of Python can be better supported more quickly. I don't guarantee that this will happen overnight, but I'd like to make a start.