Get caching to work
Closed this issue · 3 comments
Done.
Hi, thanks for working on this. Do you have any more details about how mypy caching was integrated?
As I understand it, mypy stores cached type data in *.meta.json and *.data.json files. The mypy.sh.tmpl uses the mypy --cache-map flag to specify a cache file for each source file. However, since these output files are not declared to Bazel, they simply get swallowed up by the filesystem sandbox, so they aren't reused on the next invocation of mypy.
Did I miss something? It seems like caching isn't actually improving type checking speed.
I considered simply adding the cache files as declared outputs, but there's a few more hurdles: (1) since the mypy aspect propagates along deps, we'll end up with multiple rules declaring overlapping outputs, and (2) we need to feed these output files back into the next invocation of mypy, so they need to be declared as inputs as well.
This sort of feels contrary to how Bazel expects build artifact caching to work, where a rule produces intermediate outputs (like object files) for its sources, and downstream rules get a performance speedup by ingesting these intermediate outputs rather than rebuilding them from source. It seems like mypy expects more control over its own caching, which isn't easy to adapt to Bazel. The only alternative I see is allowing mypy to manage its own cache outside of Bazel's control.