Everything About Web Programming (except Programming)

This session was the first of three where I did my PyCon duty and chaired a session. I was glad Ian Bicking's talk was included as I'd been unable to record his earlier session. Ian suggested that the focus of his talk was on "accepting my inner sysdadmin".

Imaginary Landscape is a small company with, like many small companies, vague division of responsibilities. Ian wears two hats: the programmer (who likes to write new things) and the sysadmin (who would rather redeploy well-understood software). Conway's Law applies: organizations that design systems are constrained to produce designs which are copies of the communication structures of these organizations.

The company got where they are via Zope 2 development - which "felt a lot like PHP", with no overall process. So they moved into Webware and Subversion - but again without an overall plan. Things moved into production without a definite transition.

Deployment was stressful, so they decided to stop deploying! Applications were "multi-customer", and rather than developing good tools they would build complex applications. Ian now realises this was dumb, though it seemed smarter at the time. Unfortunately it meant that since everyone got the same software customization was infeasible, and configuration data had to go into the database, with no SVN control. Deployment should be easier than that, and simple deployment obviates the need for multi-customer applications: each customer can receive their own code.

Paste is a toolset that takes advantage of standard Subversion layout, providing a skeletal setup.py file and database model and a basic framework of templates and internally-used metadata. Testing also starts with a basic model created by the tool. Functional tests are really important: even just knowing that you can access the root page of an application is a worthwhile test.

Sometimes it can be problematic developing web applications in a test-driven environment. Developing code without opening it in a browser sounds cool, but it turns out that it's too easy to overlook gaping holes in an application (like there's no link to a page that the test framework has been accessing directly). This level of purity turns out to be too extreme in practice.

Configuration data is essential to a project, and it's important to differentiate between program configuration as usedby programmers and application configuration as used by system administrators. The tool creates a template for the deployment configuratiom file. Client data is kept in separate repositories, not int he application, and controlled separately from the application code.

Deployment uses the buildutils, and installation is moving towards a two-stage process. The first step installs the configuration file and the second step sets the application up in line with that configuration. Setuptools has the ability to install multiple versions of the same product, but this turned out not to be useful, as the separation between the versions wasn't sufficiently clean.

Ian now feels that "the computer" is a very poor context for installation of anything at all. ("Site-packages considered harmful"). So each site gets its own Python environment in development, and they use sitecustomize code to configure aaccording to the ACTIVE_SITE environment variable.

A nicely-focused talk with some valuable lessons for us all.