Metaclass Madness (Python 3 Version)

Last week I was at PyCon Asia Pacific to deliver the opening keynote. Liew Beng Keat, the conference chair, was kind enough to invite me to give a technical talk as well, so I brushed up a talk that I had given previously to the Icelandic Python User Group entitled Metaclass Madness. The material is fairly straightforward, but metaclasses have the reputation for making people's heads explode, so the title was something of a warning for the unwary.

The evening before the presentation I decided to update the code, so the current download includes not only the PowerPoint slides but also usable source code for both Python 2.x and Python 3.x.

I had thought of adding class decorators to the talk, but interestingly I realized that the example I was using didn't translate. The issue was that the metaclass is called with three arguments, the third of which is a dict containing the namespace that has been constructed during the compilation of the class body. So in the metaclass's __new__() method it is easy to decorate each method by iterating over the namespace dict and replacing each callable (whose name does not begin with a double underscore) with the result of applying a decorator to it.

A class decorator cannot work this way, though (at least with a new-style class, which is all you have in Python 3). The reason is that new-style classes use a dict_proxy object as their __dict__, and the dict_proxy does not all you to set items. Consequently, by the time the decorator gets called the class __dict__ is already pretty much set in concrete.

Since the particular example I chose deliberately omitted the methods whose names began with a double underscore someone asked me whether name mangling would affect the process. [For those who don't know about name mangling it is an attempt to protect "private" variables, those whose names begin with a double underscore and end with at most one underscore. See this documentation page for further details]. I was able to demonstrate interactively, after a couple of false starts, that mangling apparently took place *after* the call to __new__() (presumably in type.__new__(), which the metaclass __new__() method must call to ensure completion of the class creation).