Error "ImportError: cannot import name 'packaging' from 'pkg_resources'" when using latest setuptools version 70
jeffhataws opened this issue · 10 comments
jeffhataws commented
When running examples in environment where the latest setuptools version 70 is installed, you may get the following error (example is with Nemo Megatron):
[CommandExecutionError] Command exited with status: 1. Error: Traceback (most recent call last):
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
main()
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 134, in get_requires_for_build_wheel
return hook(config_settings)
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup
super().run_setup(setup_script=setup_script)
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 10, in <module>
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 25, in <module>
from pkg_resources import packaging # type: ignore[attr-defined]
ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pkg_resources/__init__.py)
This is due to deprecation of pkg_resources API from setuptools since version 68: https://setuptools.pypa.io/en/latest/pkg_resources.html
The work-around is to pin setuptools version to before version 70:
pip install setuptools==69.5.1
MikeyGea commented
downgrading to setuptools 69.5.1 isn't resolving the issue. Ubuntu
jeffhataws commented
@MikeyGea, I ran the following experiments to check.
With setuptools==70:
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ pip install setuptools==70
Looking in indexes: https://pypi.org/simple, https://pip.repos.neuron.amazonaws.com
Collecting setuptools==70
Using cached setuptools-70.0.0-py3-none-any.whl.metadata (5.9 kB)
Using cached setuptools-70.0.0-py3-none-any.whl (863 kB)
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 69.5.1
Uninstalling setuptools-69.5.1:
Successfully uninstalled setuptools-69.5.1
Successfully installed setuptools-70.0.0
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from pkg_resources import packaging"
<string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pkg_resources/__init__.py)
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from torch.utils import cpp_extension"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 28, in <module>
from pkg_resources import packaging # type: ignore[attr-defined]
ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pkg_resources/__init__.py)
With setuptools==69.5.1:
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ pip install setuptools==69.5.1
Looking in indexes: https://pypi.org/simple, https://pip.repos.neuron.amazonaws.com
Collecting setuptools==69.5.1
Using cached setuptools-69.5.1-py3-none-any.whl.metadata (6.2 kB)
Using cached setuptools-69.5.1-py3-none-any.whl (894 kB)
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 70.0.0
Uninstalling setuptools-70.0.0:
Successfully uninstalled setuptools-70.0.0
Successfully installed setuptools-69.5.1
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from pkg_resources import packaging"
<string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from torch.utils import cpp_extension"
MikeyGea commented
@MikeyGea, I ran the following experiments to check.
With setuptools==70:
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ pip install setuptools==70 Looking in indexes: https://pypi.org/simple, https://pip.repos.neuron.amazonaws.com Collecting setuptools==70 Using cached setuptools-70.0.0-py3-none-any.whl.metadata (5.9 kB) Using cached setuptools-70.0.0-py3-none-any.whl (863 kB) Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 69.5.1 Uninstalling setuptools-69.5.1: Successfully uninstalled setuptools-69.5.1 Successfully installed setuptools-70.0.0 (aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from pkg_resources import packaging" <string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pkg_resources/__init__.py) (aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from torch.utils import cpp_extension" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 28, in <module> from pkg_resources import packaging # type: ignore[attr-defined] ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pkg_resources/__init__.py)With setuptools==69.5.1:
(aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ pip install setuptools==69.5.1 Looking in indexes: https://pypi.org/simple, https://pip.repos.neuron.amazonaws.com Collecting setuptools==69.5.1 Using cached setuptools-69.5.1-py3-none-any.whl.metadata (6.2 kB) Using cached setuptools-69.5.1-py3-none-any.whl (894 kB) Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 70.0.0 Uninstalling setuptools-70.0.0: Successfully uninstalled setuptools-70.0.0 Successfully installed setuptools-69.5.1 (aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from pkg_resources import packaging" <string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html (aws_neuron_venv_pytorch) ubuntu@ip-10-0-8-190:~$ python -c "from torch.utils import cpp_extension"
Doesn't matter. Installing setuptools 69.5.1 had no change in results. I think there are like 4 different python libraries and potentially infinite venvs. Ubuntu has python3 user and global, then just python. Somewhere on my system the version 70 setuptool is getting used by stable diffusion.
2019211753 commented
pip install setuptools==69.5.1 solves my trouble on Ubuntu20.04
Israel-Laguan commented
It worked on my case too, Debian 12, python 3.11, inside mamba