pip install -r requirements.txt fails
Closed this issue · 9 comments
RylanSchaeffer commented
I created a new virtual environment and immediately tried to install the requirements, but ran into a version error:
pip install -r requirements.txt
Requirement 'jaxlib @ https://storage.googleapis.com/jax-releases/cuda101/jaxlib-0.1.49-cp36-none-linux_x86_64.whl' looks like a filename, but the file does not exist
Collecting asn1crypto==0.24.0 (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
100% |████████████████████████████████| 102kB 2.9MB/s
Collecting cloudpickle==1.2.2 (from -r requirements.txt (line 3))
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/c1/49/334e279caa3231255725c8e860fa93e72083567625573421db8875846c14/cloudpickle-1.2.2-py2.py3-none-any.whl
Collecting cryptography==2.1.4 (from -r requirements.txt (line 4))
Downloading https://files.pythonhosted.org/packages/4e/e0/4959b48f04c879414972048fe2bedc96825e39c5413ae241c230fba58783/cryptography-2.1.4-cp36-cp36m-manylinux1_x86_64.whl (2.2MB)
100% |████████████████████████████████| 2.2MB 721kB/s
Collecting future==0.18.2 (from -r requirements.txt (line 5))
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829kB)
100% |████████████████████████████████| 829kB 1.6MB/s
Collecting idna==2.6 (from -r requirements.txt (line 6))
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 5.9MB/s
Collecting keyring==10.6.0 (from -r requirements.txt (line 7))
Downloading https://files.pythonhosted.org/packages/b1/4a/89ab7aa2cf501a5e715c7bbb0df11af0c0b2b1d918cdfabca74984dd2c34/keyring-10.6.0-py2.py3-none-any.whl
Collecting keyrings.alt==3.0 (from -r requirements.txt (line 8))
Downloading https://files.pythonhosted.org/packages/33/96/a2bc650d259ad510689656c50c06ae2de1299df59d369f21ec5ac0f20c8f/keyrings.alt-3.0-py2.py3-none-any.whl
Collecting opencv-python==4.2.0.34 (from -r requirements.txt (line 9))
Cache entry deserialization failed, entry ignored
Using cached https://files.pythonhosted.org/packages/72/c2/e9cf54ae5b1102020ef895866a67cb2e1aef72f16dd1fde5b5fb1495ad9c/opencv_python-4.2.0.34-cp36-cp36m-manylinux1_x86_64.whl
Collecting opt-einsum==3.2.1 (from -r requirements.txt (line 10))
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/63/a5/e6c07b08b934831ccb8c98ee335e66b7761c5754ee3cabfe4c11d0b1af28/opt_einsum-3.2.1-py3-none-any.whl (63kB)
100% |████████████████████████████████| 71kB 5.8MB/s
Collecting pycrypto==2.6.1 (from -r requirements.txt (line 11))
Downloading https://files.pythonhosted.org/packages/60/db/645aa9af249f059cc3a368b118de33889219e0362141e75d4eaf6f80f163/pycrypto-2.6.1.tar.gz (446kB)
100% |████████████████████████████████| 450kB 2.5MB/s
Collecting pyglet==1.3.2 (from -r requirements.txt (line 12))
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/1c/fc/dad5eaaab68f0c21e2f906a94ddb98175662cc5a654eee404d59554ce0fa/pyglet-1.3.2-py2.py3-none-any.whl (1.0MB)
100% |████████████████████████████████| 1.0MB 1.4MB/s
Collecting pygobject==3.26.1 (from -r requirements.txt (line 13))
Could not find a version that satisfies the requirement pygobject==3.26.1 (from -r requirements.txt (line 13)) (from versions: 3.27.0, 3.27.1, 3.27.2, 3.27.3, 3.27.4, 3.27.5, 3.28.0, 3.28.1, 3.28.2, 3.28.3, 3.29.1.dev0, 3.29.2.dev0, 3.29.3.dev0, 3.30.0, 3.30.1, 3.30.2, 3.30.3, 3.30.4, 3.30.5, 3.31.1.dev0, 3.31.2.dev0, 3.31.3.dev0, 3.31.4.dev0, 3.32.0, 3.32.1, 3.32.2, 3.33.1.dev0, 3.34.0, 3.36.0, 3.36.1, 3.38.0)
No matching distribution found for pygobject==3.26.1 (from -r requirements.txt (line 13))
RylanSchaeffer commented
This is Python 3.6.9 and pip 20.2.3
RylanSchaeffer commented
I tried removing the offending library pygobject and rerunning, but I ran into another error:
Collecting dm-haiku@ git+git://github.com/deepmind/dm-haiku.git@db991d56563221d5a06be5e7228155e53d01aba9
Cloning git://github.com/deepmind/dm-haiku.git (to revision db991d56563221d5a06be5e7228155e53d01aba9) to /tmp/pip-install-h7datbt0/dm-haiku
ERROR: Command errored out with exit status 1:
command: /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h7datbt0/dm-haiku/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h7datbt0/dm-haiku/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-uqzltyvu
cwd: /tmp/pip-install-h7datbt0/dm-haiku/
Complete output (5 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-h7datbt0/dm-haiku/setup.py", line 18, in <module>
from setuptools import find_namespace_packages
ImportError: cannot import name 'find_namespace_packages'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
RylanSchaeffer commented
A related error for dm-sonnet seemed to be caused by the user using Python 2 but I am certainly using Python3:
(dqn_zoo_venv) rylan@rylan-ThinkPad-E590:~/Documents/dqn_zoo$ pip --version
pip 20.2.3 from /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/lib/python3.6/site-packages/pip (python 3.6)
RylanSchaeffer commented
Another post suggested uninstalling and then reinstalling setuptools, which worked for me.
RylanSchaeffer commented
But now a new error appears with python-snappy:
~/Documents/dqn_zoo$ pip install -r requirements.txt
Collecting asn1crypto==0.24.0
Using cached asn1crypto-0.24.0-py2.py3-none-any.whl (101 kB)
Collecting cloudpickle==1.2.2
Using cached cloudpickle-1.2.2-py2.py3-none-any.whl (25 kB)
Collecting cryptography==2.1.4
Using cached cryptography-2.1.4-cp36-cp36m-manylinux1_x86_64.whl (2.2 MB)
Collecting future==0.18.2
Using cached future-0.18.2.tar.gz (829 kB)
Collecting idna==2.6
Using cached idna-2.6-py2.py3-none-any.whl (56 kB)
Collecting keyring==10.6.0
Using cached keyring-10.6.0-py2.py3-none-any.whl (38 kB)
Collecting keyrings.alt==3.0
Using cached keyrings.alt-3.0-py2.py3-none-any.whl (21 kB)
Collecting opencv-python==4.2.0.34
Using cached opencv_python-4.2.0.34-cp36-cp36m-manylinux1_x86_64.whl (28.2 MB)
Collecting opt-einsum==3.2.1
Using cached opt_einsum-3.2.1-py3-none-any.whl (63 kB)
Collecting pycrypto==2.6.1
Using cached pycrypto-2.6.1.tar.gz (446 kB)
Collecting pyglet==1.3.2
Using cached pyglet-1.3.2-py2.py3-none-any.whl (1.0 MB)
Collecting pyxdg==0.25
Using cached pyxdg-0.25.tar.gz (48 kB)
Collecting SecretStorage==2.3.1
Using cached SecretStorage-2.3.1.tar.gz (16 kB)
Collecting six==1.15.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting absl-py==0.9.0
Using cached absl-py-0.9.0.tar.gz (104 kB)
Collecting numpy==1.18.0
Using cached numpy-1.18.0-cp36-cp36m-manylinux1_x86_64.whl (20.1 MB)
Collecting Pillow==7.1.2
Using cached Pillow-7.1.2-cp36-cp36m-manylinux1_x86_64.whl (2.1 MB)
Collecting python-snappy==0.5.4
Using cached python-snappy-0.5.4.tar.gz (21 kB)
Collecting scipy==1.2.1
Using cached scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8 MB)
Processing /home/rylan/.cache/pip/wheels/af/49/6c/0e95549f4da2ae98ecdb63d3873a11b6b757151e0ae0ae1b9f/jax-0.1.72-py3-none-any.whl
Collecting jaxlib@ https://storage.googleapis.com/jax-releases/cuda101/jaxlib-0.1.49-cp36-none-linux_x86_64.whl
Using cached https://storage.googleapis.com/jax-releases/cuda101/jaxlib-0.1.49-cp36-none-linux_x86_64.whl (62.0 MB)
Collecting atari-py==0.2.6
Using cached atari_py-0.2.6-cp36-cp36m-manylinux1_x86_64.whl (2.8 MB)
Collecting gym==0.13.1
Using cached gym-0.13.1.tar.gz (1.6 MB)
Collecting dm-env==1.2
Using cached dm_env-1.2-py3-none-any.whl (22 kB)
Collecting dm-haiku@ git+git://github.com/deepmind/dm-haiku.git@db991d56563221d5a06be5e7228155e53d01aba9
Cloning git://github.com/deepmind/dm-haiku.git (to revision db991d56563221d5a06be5e7228155e53d01aba9) to /tmp/pip-install-3c8l7h9u/dm-haiku
Collecting dm-tree==0.1.5
Using cached dm_tree-0.1.5-cp36-cp36m-manylinux1_x86_64.whl (294 kB)
Collecting optax==0.0.1
Downloading optax-0.0.1-py3-none-any.whl (49 kB)
|████████████████████████████████| 49 kB 1.8 MB/s
Collecting rlax@ git+git://github.com/deepmind/rlax.git@870cba1ea8ad36725f4f3a790846298657b6fd4b
Cloning git://github.com/deepmind/rlax.git (to revision 870cba1ea8ad36725f4f3a790846298657b6fd4b) to /tmp/pip-install-3c8l7h9u/rlax
Collecting cffi>=1.7; platform_python_implementation != "PyPy"
Downloading cffi-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (400 kB)
|████████████████████████████████| 400 kB 6.9 MB/s
Collecting chex>=0.0.2
Downloading chex-0.0.2-py3-none-any.whl (37 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting dataclasses==0.7; python_version < "3.7"
Using cached dataclasses-0.7-py3-none-any.whl (18 kB)
Collecting toolz>=0.9.0
Downloading toolz-0.11.1-py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 4.6 MB/s
Using legacy 'setup.py install' for future, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pycrypto, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pyxdg, since package 'wheel' is not installed.
Using legacy 'setup.py install' for SecretStorage, since package 'wheel' is not installed.
Using legacy 'setup.py install' for absl-py, since package 'wheel' is not installed.
Using legacy 'setup.py install' for python-snappy, since package 'wheel' is not installed.
Using legacy 'setup.py install' for gym, since package 'wheel' is not installed.
Using legacy 'setup.py install' for dm-haiku, since package 'wheel' is not installed.
Using legacy 'setup.py install' for rlax, since package 'wheel' is not installed.
Installing collected packages: asn1crypto, cloudpickle, six, idna, pycparser, cffi, cryptography, future, SecretStorage, keyring, keyrings.alt, numpy, opencv-python, opt-einsum, pycrypto, pyglet, pyxdg, absl-py, Pillow, python-snappy, scipy, jax, jaxlib, atari-py, gym, dm-tree, dm-env, dm-haiku, dataclasses, toolz, chex, optax, rlax
Running setup.py install for future ... done
Running setup.py install for SecretStorage ... done
Running setup.py install for pycrypto ... done
Running setup.py install for pyxdg ... done
Running setup.py install for absl-py ... done
Running setup.py install for python-snappy ... error
ERROR: Command errored out with exit status 1:
command: /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5vwf1ixf/install-record.txt --single-version-externally-managed --compile --install-headers /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/include/site/python3.6/python-snappy
cwd: /tmp/pip-install-3c8l7h9u/python-snappy/
Complete output (23 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/snappy
copying snappy/__main__.py -> build/lib.linux-x86_64-3.6/snappy
copying snappy/snappy_cffi.py -> build/lib.linux-x86_64-3.6/snappy
copying snappy/snappy.py -> build/lib.linux-x86_64-3.6/snappy
copying snappy/hadoop_snappy.py -> build/lib.linux-x86_64-3.6/snappy
copying snappy/__init__.py -> build/lib.linux-x86_64-3.6/snappy
copying snappy/snappy_cffi_builder.py -> build/lib.linux-x86_64-3.6/snappy
copying snappy/snappy_formats.py -> build/lib.linux-x86_64-3.6/snappy
running build_ext
building 'snappy._snappy' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/snappy
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/rylan/Documents/dqn_zoo/dqn_zoo_venv/include -I/usr/include/python3.6m -c snappy/snappymodule.cc -o build/temp.linux-x86_64-3.6/snappy/snappymodule.o
snappy/snappymodule.cc:31:10: fatal error: snappy-c.h: No such file or directory
#include <snappy-c.h>
^~~~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5vwf1ixf/install-record.txt --single-version-externally-managed --compile --install-headers /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/include/site/python3.6/python-snappy Check the logs for full command output.
RylanSchaeffer commented
If you have an error with python-snappy, you may need to install Snappy C separately
RylanSchaeffer commented
Ok everything succeeded so long as pygobject==3.26.1 is omitted.
jqdm commented
I've clarified the purpose of requirements.txt in f6ea040. These kind of dependency issues are very common since everyone has a different set up and is precisely why a Dockerfile was provided. requirements.txt is specifically only for use with Dockerfile, and is not generally applicable.
Zekzak31 commented
python3 -m pip install-r requirements.txt