Make astroid typable possibily with breaking change to permit most performance increase possibility
Pierre-Sassoulas opened this issue · 3 comments
astroid is very dynamic and very hard to type. A lot of effort went into typing it, and mypy is still not activated (#1287). It prevents automated error detection in pylint and it prevent most performance possibility that always start with proper typing, be it mypyc, a partial rewrite in a rust crate (#2014) or cython.
I'm opening this discussion so we can discuss the breaking changes we need to do to make it easier to type astroid in the future.
Many node types have fields that are marked as optional when really they are required. These fields are assumed to be non-optional throughout the Astroid code, and that's a major source of type errors detected by Mypy.
For example, consider the Assert node. It has these fields:
self.test: NodeNG | None = None
"""The test that passes or fails the assertion."""
self.fail: NodeNG | None = None # can be None
"""The message shown when the assertion fails."""An assert statement without a test is not valid syntax, so the test field needs to be required. On the other hand, the fail message really is optional, so the fail field really is optional.
A symptom of the problem is the frequent appearance of # can be None comments. These show that the existing type annotations are not reliable.
Now, the reason AFAICT that these fields are all marked as optional is that the type annotations are declared in __init__ but the values for those fields are not available until postinit is called. There is an easy solution to this: declare the types outside of __init__. This has the happy side effect of making it possible to get rid of __init__ definitions altogether for many nodes, since all they do besides declaring types is calling super().__init__.