Type Annotations in Python

This is an edited live blog of a Python London presentation by Bernat Gabor of Bloomberg. Sorry it's taken so long to prepare. A great talk, very well presented. Thanks Bernat! Many thanks to Babylon Health for being a welcoming host.

PEP 484 introduced type hinting. Function annotations from PEP 3172 and variable annotations from PEP 526 have come together, and are supported by mypy. Why? Primarily to make code easier to debug, maintain and understand. An implementation supports annotation in Python 2.7, as structured comments, so there's no excuse for not using them!

What do annotations NOT provide?

• Runtime type inference.
• Performance improvements. 

In fact, annotations are effectively treated as commentary during Python parsing by the interpreter, other than ensuring that their syntax is valid. The mypy project can be used to verify that code calling functions conform to the type annotations thereon. You can also implement "gradual typing": the mypy linter will report errors if your code is type-hinted. It won't complain about untyped values and functions.
Type annotations as implemented by recent Python 3 releases (> 3.4) are the canonical way to do it, but require you to import all type dependencies, and the parser has to parse them. This adds a measurable time penalty, though 3.7's PEP 536 implementation will lead to increased speed.
For older Python implementations, annotations in comments will work under any Python version (simply being ignored as comments by those versions that don't understand them), but it's "kinda ugly" and creates noise around your program logic. They can also lead to compatibility problems with established linters.

A further option is to write interface or stub files - you can even create stubs for code you don't own and/or have access to the source of. This again works in any version because the stub files are simply ignored, but it requires no change to existing source and lets you use the latest Python features. It does, however, create a maintenance burden. Further, if your stubs don't match the code (if a function name is changed in the main file but not in the stub, for example), there are no checks to alert the programmer.

Any solution not using the latest syntax is likely to cause problems in the long-term, though it won't be impossible.

PEP 563, due in Python 3.7, will allow you to distribute any package with type information. Unfortunately, this will not allow annotation of local variables, only interfaces. Annotations can be incorporated into docstrings, but this isn't an especially good option, and isn't recommended.

So, what kind of types can you add? Nominal types (int, float, object, etc.) have generic containers such as Tuple[int, float], Dict[str, int], MutableMapping[str, int], List[int], Iterable[Text], and so on. Since these types are Python objects, this means you can alias types by assignment.

Further nominal types include callable and generics like TypeVar, as well as Any, which effectively disables type checking for specific names. PEP 544 will specify protocols, and this will allow the introduction of structural typing.

What are the gotchas?

• Python 2 and 3 require version-dependent checks in code intended for both environments
• It's difficult to handle functions that have multiple return types, encouraging programmers to short-circuit type checking. While the interpreter won't complain at the resulting necessary shenanigans, various linters will be unhappy.
• Type lookup can be problematic, as the system allows the programmer to shadow the names of types without realising it because they use the same scopes as the runtime.
• Subclasses can support supersets of the types supported by their superclasses, but not subsets of those types. Further, subtypes whose methods have signatures requiring additional arguments must make those additional arguments optional for type checking to succeed.

Type hinting is fun, but may cause you (like David Beazley) to wish you had a desk-side bridge to jump off. If all else fails, use # type: ignore to just shut the noise up, though this might be regarded as an admission of defeat.

PEP 257 defines how to annotate your code so Sphinx can create documentation that includes type information. Just install the sphinx-autodoc module and add it to to your conf.py file.

What's next? MyPy is getting close to a 1.0 release, and the main focus now is on improving performance and enabling incremental typing, defining and implementing an API and a plugin system to open up the ecosystem.