Crash when testing a project that has azure libraries installed
patrys opened this issue · 5 comments
Describe the bug
When attempting to test a project that uses azure-common and azure-core, deptry crashes.
To Reproduce
Steps to reproduce the behavior:
- Clone
https://github.com/saleor/saleor - Create a virtual environment and install the dependencies with
pip install -r requirements_dev.txt - Install deptry and run
deptry . - See it crash on what looks like UTF BOM:
Scanning 15104 files...
Traceback (most recent call last):
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/bin/deptry", line 8, in <module>
sys.exit(deptry())
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/deptry/cli.py", line 216, in deptry
Core(
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/deptry/core.py", line 56, in run
for mod in ImportParser().get_imported_modules_for_list_of_files(all_python_files)
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/deptry/import_parser.py", line 25, in get_imported_modules_for_list_of_files
modules_per_file = [self.get_imported_modules_from_file(file) for file in li
st_of_files]
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/deptry/import_parser.py", line 25, in <listcomp>
modules_per_file = [self.get_imported_modules_from_file(file) for file in list_of_files]
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/deptry/import_parser.py", line 38, in get_imported_modules_from_file
modules = self._get_imported_modules_from_py(path_to_file)
File "/Users/patrys/GitHub/saleor/saleor/.direnv/python-3.9.15/lib/python3.9/site-packages/deptry/import_parser.py", line 55, in _get_imported_modules_from_py
root = ast.parse(f.read(), path_to_py_file) # type: ignore[call-overload]
File "/usr/local/Cellar/python@3.9/3.9.15/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File ".direnv/python-3.9.15/lib/python3.9/site-packages/azure/common/__init__.py", line 1
#-------------------------------------------------------------------------
^
SyntaxError: invalid non-printable character U+FEFF
Expected behavior
Well, I'd like it to work.
System [please complete the following information]:
- OS: MacOS Ventura 13.0
- Language Version: cPython 3.9.15
- Poetry version: 1.2.2
Additional context
N/A
@patrys Thanks for raising the issue. When following your steps, deptry runs successfully for me. The main difference seems to be the files that are scanned: For me it shows Scanning 2184 files... while I see Scanning 15104 files... in your example.
Is your virtual environment stored in a folder with a different name than venv or .venv? When I tried to reproduce your example, I created the virtual environment in a folder called venv, which is one of the directories that is ignored by default, see here or this section of the deptry --help command:
-e, --exclude TEXT A regular expression for directories or
files in which .py files should not be
scanned for imports to determine if there
are dependency issues. Can be used multiple
times by specifying the argument multiple
times. re.match() is used to match the
expressions, which by default checks for a
match only at the beginning of a string. For
example: `deptry . -e ".*/foo/" -e bar"`
Note that this overwrites the defaults.
[default: venv, \.venv, tests, \.git,
setup.py]
-ee, --extend-exclude TEXT Like --exclude, but adds additional files
and directories on top of the excluded ones
instead of overwriting the defaults. (Useful
if you simply want to add to the default)
`deptry . -ee ".*/foo/" -ee bar"`
If that is not the case, are there any other directories that you have locally within your project that are not in the list of defaults for exclude? If so, run deptry with the -ee flag, or add a section [tool.deptry] to your pyproject.toml. More details about that can be found here.
If you are unsure, you can run deptry -v . which will log all the files that are scanned to the console.
Hope this helps!
@patrys Now I take a better look; it seems your virtual environment in stored in .direnv, is that correct? In that case, can you try running deptry . -ee ".direnv"?
FYI, the output I see for your project with the default settings is the following:
There were 32 dependency issues found.
-----------------------------------------------------
The project contains obsolete dependencies:
azure-common
azure-storage-blob
azure-storage-common
django-redis
django-storages
gunicorn
maxminddb
psycopg2
python-magic-bin
Consider removing them from your project's dependencies. If a package is used for development purposes, you should add it to your development dependencies instead.
-----------------------------------------------------
There are dependencies missing from the project's list of dependencies:
PIL
dateutil
graphql
magic
pkg_resources
Consider adding them to your project's dependencies.
-----------------------------------------------------
There are transitive dependencies that should be explicitly defined as dependencies:
Rx
asgiref
botocore
graphql-relay
kombu
lxml
measurement
mock
promise
python-http-client
pytz
redis
urllib3
They are currently imported but not specified directly as your project's dependencies.
-----------------------------------------------------
There are imported modules from development dependencies detected:
before-after
django-debug-toolbar
django-graphiql-debug-toolbar
fakeredis
pytest
Consider moving them to your project's 'regular' dependencies. If this is not correct and the dependencies listed above are indeed development dependencies, it's likely that files were scanned that are only used for development purposes. Run `deptry -v .` to see a list of scanned files.
-----------------------------------------------------
Dependencies and directories can be ignored by passing additional command-line arguments. See `deptry --help` for more details.
Alternatively, deptry can be configured through `pyproject.toml`. An example:
```
[tool.deptry]
ignore_obsolete = [
'foo'
]
ignore_missing = [
'bar'
]
ignore_transitive = [
'baz'
]
exclude = [
'venv','.venv', 'tests', 'setup.py', 'docs'
]
```
For more information, see the documentation: https://fpgmaas.github.io/deptry/
If you have encountered a bug, have a feature request or if you have any other feedback, please file a bug report at https://github.com/fpgmaas/deptry/issues/new/choose