mypy 0.780 source file found twice under different module names error
Opened this issue · 6 comments
This bazel plugin works well with mypy 0.770 but after upgrading to 0.780 it complains with
"source file found twice under different module names error". mypy 0.780 works fine outside of bazel.
There is a related issue on mypy: python/mypy#8944. In my case, directories on mypy_path
do not contain __init__.py
files.
- I am setting the
mypy_path
withinmypy.ini
. Even if I disable that, the problem persists. - Bazel tests pass if I disable the bazel mypy checks (so I think the bazel setup is - mostly - correct).
- The template adds
$(pwd)
to the front of MYPYPATH. I tried playing around with that but still hitting the same issue.
So, I am not very sure where the problem comes from.
EDIT: usually I am setting PYTHONPATH in .bazelrc and mypy_path in mypy.ini but even if I add imports = ["<path e.g. ..>"]
in all BUILD files the above steps still apply.
I just ran the aspect with 0.780
and didn't get that error (things worked fine). Can you provide a minimal reproduction, or pointers towards likely sources of the problem?
This is interesting to me:
usually I am setting PYTHONPATH in .bazelrc
How exactly are you doing that, and why? (--action-env
?)
Yes, I am using --action-env
to set PYTHONPATH simply because I noticed it works and it reduces the recurring imports= ...
so it is easier.
My repo looks like:
monorepo/
WORKSPACE
mypy.ini
python_package_1/
BUILD
setup.py
my_first_package/
BUILD
__init__.py
subpackage/
....
python_package_2/
BUILD
setup.py
my_second_package/
<similar to my_first_package>
...
In mypy.ini I set mypy_path = my_first_package:my_second_package
.
mypy complains:
error: Source file found twice under different module names: 'python_package_1.my_first_package....x.py' and 'my_first_package....x.py'
If I go to mypy.sh.tpl
and set --package-root
to python_package_1
the error goes away, tests get to run and pass. However, there is no such thing like package roots - note the plural (even though Guido had it in plan initially).
Code to reproduce the error:
File structure:
WORKSPACE
mypy.ini
mypy_version.txt
requirements.txt (empty)
BUILD.bazel (empty)
mylib/
BUILD.bazel (empty)
welltyped/
BUILD.bazel
__init__.py
bazelized/
__init__.py
x.py
Contents:
- .bazelrc (not sure if it matters but will put it here)
startup --output_user_root=<some dir>
build --disk_cache=<some other dir>
build --aspects @mypy_integration//:mypy.bzl%mypy_aspect
build --output_groups=+mypy
build --action_env=PYTHONPATH=mylib
- mypy.ini
[mypy]
ignore_missing_imports = True
mypy_path = mylib
-
mypy_version.txt
mypy==0.780
!! Change to 0.770 and the error goes away -
welltyped/BUILD.bazel
load("@rules_python//python:defs.bzl", "py_library")
py_library(
name = "bazelized",
srcs = glob(["bazelized/*.py"]),
deps = [],
)
- welltyped/bazelized/x.py
def f(x: int) -> int:
return x
Now the strange part:
- welltyped/bazelized/__init__.py
# Expose f as "from welltyped.bazelized import f"
from welltyped.bazelize.x import f # !!! Comment this line and the error no longer appears
ERROR on bazel build mylib/welltyped:bazelized
:
mylib/welltyped/bazelized/x.py: error: Source file found twice under different module names: 'mylib.welltyped.bazelized.x' and 'welltyped.bazelized.x'
- mypy alone (outside bazel) works just fine
- mypy 0.770 works fine
- mypy 0.780 breaks only when doing that import in
__init__.py
- setting
--package-root
tomylib
instead of.
inmypy.sh.tpl
fixes the error
output=$({MYPY_EXE} {VERBOSE_OPT} --bazel --package-root mylib --config-file {MYPY_INI_PATH} --cache-map {CACHE_MAP_TRIPLES} -- {SRCS} 2>&1)
The problem with --package-root is that it only allows one root AFAIK.
Thanks for reproduction. I'll take a look.
Hmm. I'm trying to figure out the solution here. Collect up the imports
of each python dependency, and use that for mypy_path
? Otherwise, it seems like one is basically out of luck if they don't use WORKSPACE root on the PYTHONPATH.
#29 addresses this by using -m
arguments for modules rather than source files. That way, source files and module paths that they're imported with are matched up, which is what mypy wants. More here: https://github.com/dshahbaz/bazel-mypy-integration/wiki#improvement-3-imports-path-aware-mypy-invocations