qiling fails to install due to an error installing pyelftools - No such file or directory: '/usr/local/lib/python3.8/dist-packages/pyelftools-0.29.dist-info/METADATA'"
Closed this issue · 4 comments
seanthegeek commented
pip_|-remnux-python3-packages-qiling_|-qiling_|-installed:
__id__: remnux-python3-packages-qiling
__run_num__: 387
__sls__: remnux.python3-packages.qiling
changes: {}
comment: "Failed to install packages: qiling. Error: Requirement already satisfied:\
\ qiling in /usr/local/lib/python3.8/dist-packages (1.4.5)\nCollecting qiling\n\
\ Obtaining dependency information for qiling from https://files.pythonhosted.org/packages/7b/ac/3b72bd96612a2d3af84426533c915de847e481cb6819be824eb967ca6ca9/qiling-1.4.6-py3-none-any.whl.metadata\n\
\ Downloading qiling-1.4.6-py3-none-any.whl.metadata (12 kB)\nRequirement already\
\ satisfied: capstone>=4.0.1 in /usr/local/lib/python3.8/dist-packages (from\
\ qiling) (4.0.2)\nCollecting unicorn>=2.0.1 (from qiling)\n Downloading unicorn-2.0.1.post1-py2.py3-none-manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl\
\ (16.1 MB)\n ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.1/16.1 MB 14.5\
\ MB/s eta 0:00:00\nRequirement already satisfied: pefile>=2022.5.30 in /usr/local/lib/python3.8/dist-packages\
\ (from qiling) (2023.2.7)\nRequirement already satisfied: python-registry>=1.3.1\
\ in /usr/local/lib/python3.8/dist-packages (from qiling) (1.3.1)\nRequirement\
\ already satisfied: keystone-engine>=0.9.2 in /usr/local/lib/python3.8/dist-packages\
\ (from qiling) (0.9.2)\nRequirement already satisfied: pyelftools>=0.28 in\
\ /usr/local/lib/python3.8/dist-packages (from qiling) (0.29)\nRequirement already\
\ satisfied: gevent>=20.9.0 in /usr/local/lib/python3.8/dist-packages (from\
\ qiling) (21.12.0)\nRequirement already satisfied: multiprocess>=0.70.12.2\
\ in /usr/local/lib/python3.8/dist-packages (from qiling) (0.70.12.2)\nRequirement\
\ already satisfied: pyyaml>=6.0 in /usr/local/lib/python3.8/dist-packages (from\
\ qiling) (6.0.1)\nRequirement already satisfied: python-fx in /usr/local/lib/python3.8/dist-packages\
\ (from qiling) (0.2.0)\nRequirement already satisfied: questionary in /usr/local/lib/python3.8/dist-packages\
\ (from qiling) (1.10.0)\nRequirement already satisfied: termcolor in /usr/local/lib/python3.8/dist-packages\
\ (from qiling) (1.1.0)\nRequirement already satisfied: zope.event in /usr/local/lib/python3.8/dist-packages\
\ (from gevent>=20.9.0->qiling) (4.5.0)\nRequirement already satisfied: zope.interface\
\ in /usr/local/lib/python3.8/dist-packages (from gevent>=20.9.0->qiling) (6.0)\n\
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages\
\ (from gevent>=20.9.0->qiling) (60.9.3)\nRequirement already satisfied: greenlet<2.0,>=1.1.0\
\ in /usr/local/lib/python3.8/dist-packages (from gevent>=20.9.0->qiling) (1.1.2)\n\
Requirement already satisfied: dill>=0.3.4 in /usr/local/lib/python3.8/dist-packages\
\ (from multiprocess>=0.70.12.2->qiling) (0.3.4) ERROR: Could not install packages\
\ due to an OSError: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/pyelftools-0.29.dist-info/METADATA'"
duration: 2299.226
name: qiling
result: false
start_time: '13:04:04.884403'
seanthegeek commented
Weird. Even after upgrading setuptools pip and wheel as I described here, it still won't install.
digitalsleuth commented
Everything actually worked correctly, the error was a temporary domain resolution failure with github.com:
Line 27113: # [ERROR ] Command '/usr/bin/python3' failed with return code: 1
Line 27114: # [ERROR ] stdout: Collecting git+https://github.com/obsidianforensics/unfurl.git
Line 27116: # [ERROR ] stderr: Running command git clone --filter=blob:none --quiet https://github.com/obsidianforensics/unfurl.git /tmp/pip-req-build-af9zi5um
Cloning https://github.com/obsidianforensics/unfurl.git to /tmp/pip-req-build-af9zi5um
# [ERROR ] stderr: Running command git clone --filter=blob:none --quiet https://github.com/obsidianforensics/unfurl.git /tmp/pip-req-build-af9zi5um
fatal: unable to access 'https://github.com/obsidianforensics/unfurl.git/': Could not resolve host: github.com
error: subprocess-exited-with-error
× git clone --filter=blob:none --quiet https://github.com/obsidianforensics/unfurl.git /tmp/pip-req-build-af9zi5um did not run successfully.
│ exit code: 128
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× git clone --filter=blob:none --quiet https://github.com/obsidianforensics/unfurl.git /tmp/pip-req-build-af9zi5um did not run successfully.
│ exit code: 128
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Line 27132: # [ERROR ] retcode: 1
Line 27133: # [ERROR ] Failed to install packages: git+https://github.com/obsidianforensics/unfurl.git. Error: Collecting git+https://github.com/obsidianforensics/unfurl.gitseanthegeek commented
D'oh! I don't know how I missed that for unfurl. After I tried the upgrade again it was fine.
That said, I forgot to mention what I had to do to get the Python packages to install correctly first.
Even when using --force-reinstall, pip would still install an outdated version of setuptools
sudo pip install --force-reinstall --break-system-packages -U pip setuptools wheel
Collecting pip
Obtaining dependency information for pip from https://files.pythonhosted.org/packages/50/c2/e06851e8cc28dcad7c155f4753da8833ac06a5c704c109313b8d5a62968a/pip-23.2.1-py3-none-any.whl.metadata
Downloading pip-23.2.1-py3-none-any.whl.metadata (4.2 kB)
Collecting setuptools
Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Downloading setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
Downloading wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 8.6 MB/s eta 0:00:00
Downloading setuptools-68.2.2-py3-none-any.whl (807 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 807.9/807.9 kB 16.6 MB/s eta 0:00:00
Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 kB 15.7 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip
Attempting uninstall: wheel
Found existing installation: wheel 0.36.2
Uninstalling wheel-0.36.2:
Successfully uninstalled wheel-0.36.2
Attempting uninstall: setuptools
Found existing installation: setuptools 60.7.1
Uninstalling setuptools-60.7.1:
Successfully uninstalled setuptools-60.7.1
Attempting uninstall: pip
Found existing installation: pip 23.2
Uninstalling pip-23.2:
Successfully uninstalled pip-23.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
launchpadlib 1.10.13 requires testresources, which is not installed.
speakeasy-emulator 1.5.11 requires unicorn==1.0.2, but you have unicorn 2.0.1.post1 which is incompatible.
thug 5.0.1 requires charset-normalizer==3.1.0, but you have charset-normalizer 2.1.1 which is incompatible.
thug 5.0.1 requires cssutils==2.6.0, but you have cssutils 2.7.1 which is incompatible.
thug 5.0.1 requires requests-futures==1.0.0, but you have requests-futures 1.0.1 which is incompatible.
Successfully installed pip-23.2.1 setuptools-60.8.0 wheel-0.41.2
It turns out that I somehow had multiple versions of setuptools installed
ls /usr/local/lib/python3.8/dist-packages/setuptools
setuptools/ setuptools-68.0.0.dist-info/
setuptools-61.3.0.dist-info/ setuptools-68.2.2.dist-info/
setuptools-65.7.0.dist-info/ setuptools_rust/
setuptools-67.8.0.dist-info/ setuptools_rust-1.7.0.dist-info/
Once I manually removed those, I was able to upgrade setuptools
sudo rm -rf /usr/local/lib/python3.8/dist-packages/setuptools-*
sudo pip install --break-system-packages -U pip setuptools wheel
sudo pip install --force-reinstall --break-system-packages -U pip setuptools wheel
Collecting pip
Obtaining dependency information for pip from https://files.pythonhosted.org/packages/50/c2/e06851e8cc28dcad7c155f4753da8833ac06a5c704c109313b8d5a62968a/pip-23.2.1-py3-none-any.whl.metadata
Using cached pip-23.2.1-py3-none-any.whl.metadata (4.2 kB)
Collecting setuptools
Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
Using cached wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Using cached pip-23.2.1-py3-none-any.whl (2.1 MB)
Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
Installing collected packages: wheel, setuptools, pip
Attempting uninstall: wheel
Found existing installation: wheel 0.41.2
Uninstalling wheel-0.41.2:
Successfully uninstalled wheel-0.41.2
Attempting uninstall: setuptools
Found existing installation: setuptools 52.0.0
Uninstalling setuptools-52.0.0:
Successfully uninstalled setuptools-52.0.0
Attempting uninstall: pip
Found existing installation: pip 23.2.1
Uninstalling pip-23.2.1:
Successfully uninstalled pip-23.2.1
It was a similar situation with pyelftools
ls /usr/local/lib/python3.8/dist-packages/pyelftools*
pyelftools-0.29.dist-info/ pyelftools-0.30.dist-info/
sudo rm -rf /usr/local/lib/python3.8/dist-packages/pyelftools-*
sudo pip install --force-reinstall --break-system-packages -U pyelftools
I removed unneeded system packages.
sudo apt autoremove -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
python3-contextvars python3-croniter python3-gnupg
python3-immutables python3-jinja2 python3-markupsafe
python3-msgpack python3-pycryptodome python3-tz python3-zmq
0 upgraded, 0 newly installed, 10 to remove and 0 not upgraded.
After this operation, 22.8 MB disk space will be freed.
(Reading database ... 406430 files and directories currently instal
led.)
Removing python3-contextvars (2.4-1) ...
Removing python3-croniter (0.3.29-2ubuntu1) ...
Removing python3-gnupg (0.4.5-2) ...
Removing python3-immutables (0.14-1) ...
Removing python3-jinja2 (2.11.3-1ppa1~focal) ...
Removing python3-markupsafe (1.1.1-1ppa1~focal) ...
Removing python3-msgpack (0.6.2-1) ...
Removing python3-pycryptodome (3.6.1-2build4) ...
Removing python3-tz (2022.1-1ppa1~focal) ...
Removing python3-zmq (23.2.0-1ppa1~focal) ..
Then I was able to sucesssfully run remnux uograde
digitalsleuth commented
@seanthegeek That's good to hear. I've just discovered that setuptools 68+ has introduced a new set of problems, so the version will be downgrading in the future to resolve this (and the inability to uninstall).
Cheers!