Marius started out by outlining the reasonable uses of doctest, both inline in programs and in files that are separate from the source code, and advised us not, for example, to put complicated setup or tests for obscure corner cases in the README. DocTest can be easily integrated with unittest, by defining a test suite that runs the doctests!
He then pointed out DocTestSuite and DocFileSuite for externalising the tests, and considered how to handle complicated setup. One way is to define setup and teardown functions, and you can also put code in test.globs, but these methods should be limited for ease of comprehension.
The advantage of doctest is in encouraging simlicity and interspersed narrative to explain the tests. Another option is to define a test module. The advice is to avoid mixing long tests and regular code in the same file.
In real life, of course, some features of doctest are difficult to use when data is not reliably repeatable (such as the addresses of objects in repr()-style representations). Also problematic are ellipsis-elided outputs, which are to be ignored, and ordering issues. Web testing is even more difficult, as responses are frequently state-dependent. marcus showed some examples of testbrowser, a simple system allowing web testing.
Doctest disadvantages include the disability to step through tests. Documentation (docstrings, README files) should be separated from unit and functional tests.
Questioning revealed that Zope uses a slightly different version of doctest from the Python 2.4 distribution, although "in theory" they offer the same functionality. Jim Fulton asked whether the speaker had looked at Fit (and apprently PyFit is available) but although Jim would like to see Fit and doctest integrated there was no experience with the packages in the room.
The only slightly problematic aspect of this presentation was a couple of times when the speaker made inaudible responses to inaudible questions from the chair. This would have been less troublesome if I had not been at the back of the room, or if the PA system had allowed audibility. Otherwise I got a lot of information in a very short time, well presented.
"Any sufficiently advanced technology is indistinguishable from magic" -- Arthur C Clarke
Musings from the technological frontier
4 comments:
It is Marius not Marcus
I think you mean "Marius". :)
Thanks for pointing out the slides, Marius. It appears I should be more careful with people's names when I blog from conferences ... sorry: as you can see I've corrected it now. Thanks for your talk.
When I need to step through code in a doctest, I use:
>>> import pdb;pdb.set_trace(); foo()
And that drops me into the debugger. I can also stick a set_trace() anywhere in the code that's being tested, same deal.
I'd love to see a version of Fit that ran doctests instead of using HTML tables, and colorized its output, especially if it treated the doctest as reStructuredText. That would be cool, especially if I could just keep hitting reload in a web browser as I edited code and tests. Too bad I don't have time at the moment...
Post a Comment