[BUG] mysqlclient installation (via salt-pip) fails on 3006.6
lburinov73 opened this issue · 20 comments
Description
Trying to install mysqlclient into 3006.6 relenv fails:
salt-pip install mysqlclient
Downloading mysqlclient-2.2.1.tar.gz (89 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.0/90.0 kB 7.4 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
Error in sitecustomize; set PYTHONVERBOSE for traceback:
TypeError: expected str, bytes or os.PathLike object, not NoneType
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "/opt/saltstack/salt/lib/python3.10/site.py", line 617, in <module>
main()
File "/opt/saltstack/salt/lib/python3.10/site.py", line 610, in main
execsitecustomize()
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 932, in wrapper
import sitecustomize
File "/tmp/pip-build-env-8sua0kqf/site/sitecustomize.py", line 23, in <module>
site.addsitedir(path)
File "/opt/saltstack/salt/lib/python3.10/site.py", line 216, in addsitedir
sitedir, sitedircase = makepath(sitedir)
File "/opt/saltstack/salt/lib/python3.10/site.py", line 97, in makepath
dir = os.path.join(*paths)
File "/opt/saltstack/salt/lib/python3.10/posixpath.py", line 76, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
No problems on 3006.5
Setup
AlmaLinux 8.9, salt-minion 3006.6 installed via bootstrap.saltstack.com
A similar issue #65972
Confirmed here as well; occurred for me with libvirt-python.
Issue still occurs even if calling salt-pip
explicitly at the shell on the minion.
Running /opt/saltstack/salt/bin/python3.10 -m pip install libvirt-python
as root did work for me (after the failed pip.installed state ran, at least). You'll probably have to restart salt-minion after.
This definitely looks like a salt-pip/pip.installed
cockup.
I can confirm the issue and that /opt/saltstack/salt/bin/python3.10 -m pip install mysqlclient
is a workaround for my version:
Salt Version:
Salt: 3006.6
Python Version:
Python: 3.10.13 (main, Nov 15 2023, 04:34:27) [GCC 11.2.0]
Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.3
libgit2: Not Installed
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.14.2
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: debian 12 bookworm
locale: utf-8
machine: x86_64
release: 6.1.0-18-amd64
system: Linux
version: Debian GNU/Linux 12 bookworm
Confirming this still occurs in 3006.7 per milestone set above.
Still occurs in 3006.7, the workaround of /opt/saltstack/salt/bin/python3.10 -m pip install mysqlclient
works
I had the same issue (using Debian bookworm, salt 3006.7, pip 23.3.2|| pip 24.0); and the /opt/saltstack/salt/bin/python3.10 -m pip install mysqlclient
didn't work through this.
Collecting mysqlclient
Using cached mysqlclient-2.2.4.tar.gz (90 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [27 lines of output]
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
Trying pkg-config --exists mysqlclient
Command 'pkg-config --exists mysqlclient' returned non-zero exit status 127.
Trying pkg-config --exists mariadb
Command 'pkg-config --exists mariadb' returned non-zero exit status 127.
Trying pkg-config --exists libmariadb
Command 'pkg-config --exists libmariadb' returned non-zero exit status 127.
Traceback (most recent call last):
File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-cq4k6mm_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-cq4k6mm_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-cq4k6mm_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 155, in <module>
File "<string>", line 49, in get_config_posix
File "<string>", line 28, in find_package_name
Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
I used this link to download dependencies and also downgraded the pip then it worked:
apt-get install python3-dev default-libmysqlclient-dev build-essential pkg-config python3-mysqldb
salt-pip install pip==22.3.1
salt-pip install mysqlclient
While I'm hesitant to downgrade the onedir pip for my machines in prod, it's definitely interesting that worked for you and may be related to an interface change between 22.3.1 and whatever onedir is distributing in 3006.6/3006.7.
Has anyone had a chance to confirm/reproduce in 3007.0 yet? I haven't turned up any new minions that use salt-pip on it yet.
I can reproduce on Salt 3007.0 on Debian Bookworm 12 amd64.
TLDR: Solution is to patch the /usr/bin/salt-pip
and /opt/saltstack/salt/salt-pip
. See at the bottom of the comment for the patch
file 😃.
salt-pip
is installed in 2 executables:
- The first one installed in
/usr/bin/salt-pip
- The second one installed in
/opt/saltstack/salt/salt-pip
root@HOST:~# dpkg -S /usr/bin/salt-pip
salt-common: /usr/bin/salt-pip
root@HOST:~# dpkg -S /opt/saltstack/salt/salt-pip
salt-common: /opt/saltstack/salt/salt-pip
Of course the one in /usr/bin/salt-pip
takes precedence:
root@HOST:~# which salt-pip
/usr/bin/salt-pip
In my case the issue occured trying to match libgit2
and python3-pygit2
(or just pygit2
when using pip
).
The first time I used salt-pip
, this failed:
root@HOST:~# salt-pip install pygit2==1.5.0 --no-deps
Collecting pygit2==1.5.0
Using cached pygit2-1.5.0.tar.gz (255 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Error in sitecustomize; set PYTHONVERBOSE for traceback:
AssertionError:
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "/opt/saltstack/salt/lib/python3.10/site.py", line 617, in <module>
main()
File "/opt/saltstack/salt/lib/python3.10/site.py", line 610, in main
execsitecustomize()
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 959, in wrapper
import sitecustomize
File "/tmp/pip-build-env-6lio8z_s/site/sitecustomize.py", line 22, in <module>
assert not path in sys.path
AssertionError
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
This worked, as mentioned by the other reporters:
root@HOST:~# /opt/saltstack/salt/bin/python3.10 -m pip install pygit2==1.5.0
Collecting pygit2==1.5.0
Using cached pygit2-1.5.0.tar.gz (255 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cffi>=1.4.0 in /opt/saltstack/salt/extras-3.10 (from pygit2==1.5.0) (1.16.0)
Collecting cached-property (from pygit2==1.5.0)
Downloading cached_property-1.5.2-py2.py3-none-any.whl.metadata (11 kB)
Requirement already satisfied: pycparser in /opt/saltstack/salt/lib/python3.10/site-packages (from cffi>=1.4.0->pygit2==1.5.0) (2.21)
Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Building wheels for collected packages: pygit2
Building wheel for pygit2 (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for pygit2 (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [64 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/credentials.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/packbuilder.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/submodule.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/repository.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/_run.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/_build.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/utils.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/settings.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/errors.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/remote.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/__init__.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/blame.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/callbacks.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/ffi.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/index.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/refspec.py -> build/lib.linux-x86_64-cpython-310/pygit2
copying pygit2/config.py -> build/lib.linux-x86_64-cpython-310/pygit2
creating build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/merge.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/describe.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/net.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/indexer.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/checkout.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/blame.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/buffer.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/attr.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/oid.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/stash.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/pack.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/remote.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/strarray.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/submodule.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/graph.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/proxy.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/clone.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/types.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/refspec.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/errors.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/repository.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/config.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/callbacks.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/transport.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/common.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/diff.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/index.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
copying pygit2/decl/revert.h -> build/lib.linux-x86_64-cpython-310/pygit2/decl
running build_ext
generating cffi module 'build/temp.linux-x86_64-cpython-310/pygit2._libgit2.c'
creating build/temp.linux-x86_64-cpython-310
building 'pygit2._pygit2' extension
creating build/temp.linux-x86_64-cpython-310/src
x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/local/include -I/opt/saltstack/salt/include/python3.10 -c src/blob.c -o build/temp.linux-x86_64-cpython-310/src/blob.o
In file included from src/blob.c:30:
src/blob.h:33:10: fatal error: git2.h: No such file or directory
33 | #include <git2.h>
| ^~~~~~~~
compilation terminated.
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pygit2
Failed to build pygit2
ERROR: Could not build wheels for pygit2, which is required to install pyproject.toml-based projects
(ignore the error in the second command, I was missing other dependencies)
Now if I try to uninstall the pygit2
module, then reinstall it, both methods work.
Weird ...
I assume the /opt/saltstack/salt/bin/python3.10
could probably be replaced by /opt/saltstack/salt/salt-bin
although I cannot prove it (since subsequent install/uninstall of the module works correctly, no matter what).
I attempted to do the same with libvirt-python
since I basically got only "one shot" to replicate ...
root@HOST:~# salt-pip install libvirt-python
Collecting libvirt-python
Using cached libvirt-python-10.2.0.tar.gz (246 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Error in sitecustomize; set PYTHONVERBOSE for traceback:
AssertionError:
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "/opt/saltstack/salt/lib/python3.10/site.py", line 617, in <module>
main()
File "/opt/saltstack/salt/lib/python3.10/site.py", line 610, in main
execsitecustomize()
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 959, in wrapper
import sitecustomize
File "/tmp/pip-build-env-xcfido__/site/sitecustomize.py", line 22, in <module>
assert not path in sys.path
AssertionError
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
root@HOST:~# /opt/saltstack/salt/salt-pip install libvirt-python
Collecting libvirt-python
Using cached libvirt-python-10.2.0.tar.gz (246 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Error in sitecustomize; set PYTHONVERBOSE for traceback:
AssertionError:
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "/opt/saltstack/salt/lib/python3.10/site.py", line 617, in <module>
main()
File "/opt/saltstack/salt/lib/python3.10/site.py", line 610, in main
execsitecustomize()
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 959, in wrapper
import sitecustomize
File "/tmp/pip-build-env-1jnr93_p/site/sitecustomize.py", line 22, in <module>
assert not path in sys.path
AssertionError
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
root@HOST:~# /opt/saltstack/salt/bin/python3.10 -m pip install libvirt-python
Collecting libvirt-python
Using cached libvirt-python-10.2.0.tar.gz (246 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: libvirt-python
Building wheel for libvirt-python (pyproject.toml) ... done
Created wheel for libvirt-python: filename=libvirt_python-10.2.0-cp310-cp310-linux_x86_64.whl size=502444 sha256=b961d6d1c3d3be2bdbeb54dba2f63b495f07a15d59f7a8835d18eaf464afbf2b
Stored in directory: /root/.cache/pip/wheels/5b/41/9b/8fc0e14c789b09a95b0db2581e139cf446881b782a54049d75
Successfully built libvirt-python
Installing collected packages: libvirt-python
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libglib-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgobject-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgio-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libxml2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libacl.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libaudit.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcap-ng.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgnutls.so.30 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnl-3.so.200 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnuma.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libselinux.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libapparmor.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libyajl.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh2.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libsasl2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtirpc.so.3 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpcre2-8.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libmount.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicuuc.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libp11-kit.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libidn2.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libunistring.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtasn1.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnettle.so.8 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libhogweed.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmp.so.10 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnghttp2.so.14 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/librtmp.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpsl.so.5 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libldap-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/liblber-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libzstd.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlidec.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libblkid.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicudata.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libstdc++.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcom_err.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlicommon.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libkeyutils.so.1 is not in /opt/saltstack/salt
Do not adjust rpath of /opt/saltstack/salt/lib/python3.10/site-packages/libvirtmod.cpython-310-x86_64-linux-gnu.so
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt-lxc.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libapparmor.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libselinux.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libglib-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgobject-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgio-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libxml2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libacl.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libaudit.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcap-ng.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgnutls.so.30 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnl-3.so.200 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnuma.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libyajl.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh2.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libsasl2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtirpc.so.3 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpcre2-8.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libmount.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicuuc.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libp11-kit.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libidn2.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libunistring.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtasn1.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnettle.so.8 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libhogweed.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmp.so.10 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnghttp2.so.14 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/librtmp.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpsl.so.5 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libldap-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/liblber-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libzstd.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlidec.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libblkid.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicudata.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libstdc++.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcom_err.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlicommon.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libkeyutils.so.1 is not in /opt/saltstack/salt
Do not adjust rpath of /opt/saltstack/salt/lib/python3.10/site-packages/libvirtmod_lxc.cpython-310-x86_64-linux-gnu.so
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt-qemu.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libglib-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgobject-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgio-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libxml2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libacl.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libaudit.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcap-ng.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgnutls.so.30 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnl-3.so.200 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnuma.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libselinux.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libapparmor.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libyajl.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh2.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libsasl2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtirpc.so.3 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpcre2-8.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libmount.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicuuc.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libp11-kit.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libidn2.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libunistring.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtasn1.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnettle.so.8 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libhogweed.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmp.so.10 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnghttp2.so.14 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/librtmp.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpsl.so.5 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libldap-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/liblber-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libzstd.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlidec.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libblkid.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicudata.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libstdc++.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcom_err.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlicommon.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libkeyutils.so.1 is not in /opt/saltstack/salt
Do not adjust rpath of /opt/saltstack/salt/lib/python3.10/site-packages/libvirtmod_qemu.cpython-310-x86_64-linux-gnu.so
Successfully installed libvirt-python-10.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Same result:
salt-pip
fails (/usr/bin/salt-pip
)/opt/saltstack/salt/salt-pip install libvirt-python
fails/opt/saltstack/salt/bin/python3.10 -m pip install libvirt-python
works
To be able to go beyond "one shot" and be able to replicate, note the message about the /root/.cache
directory used by pip
or salt-pip
.
Between every attempt, remove that folder, then it will fail again:
rm -rf /root/.cache/pip/
Then:
root@HOST:~# /opt/saltstack/salt/salt-pip install libvirt-python
Collecting libvirt-python
Downloading libvirt-python-10.2.0.tar.gz (246 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.0/247.0 kB 4.9 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Error in sitecustomize; set PYTHONVERBOSE for traceback:
AssertionError:
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "/opt/saltstack/salt/lib/python3.10/site.py", line 617, in <module>
main()
File "/opt/saltstack/salt/lib/python3.10/site.py", line 610, in main
execsitecustomize()
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 959, in wrapper
import sitecustomize
File "/tmp/pip-build-env-7sphrlgi/site/sitecustomize.py", line 22, in <module>
assert not path in sys.path
AssertionError
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Now, I do NOT want to change the System PATH variable on a global basis, as that would change the "default" python3 installation to /opt/saltstack/salt
as well and could potentially introduce breakages.
I tried to modify /opt/saltstack/salt/salt-pip
by adding an export of the PATH variable to include both python
and pip
:
#!/bin/sh
"true" ''''
"export" "PATH=/opt/saltstack/salt/bin:/opt/saltstack/salt/extras-3.10/bin:$PATH:/opt/saltstack/salt/extras-3.10/bin:/opt/saltstack/salt/bin"
"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
'''
# -*- coding: utf-8 -*-
import re
import sys
from salt.scripts import salt_pip
if __name__ == '__main__':
print(sys.path)
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(salt_pip())
However the PATH doesn't match what I specified (I guess that's the dirname
fault):
['/opt/saltstack/salt', '/opt/saltstack/salt/extras-3.10', '/opt/saltstack/salt/lib/python310.zip', '/opt/saltstack/salt/lib/python3.10', '/opt/saltstack/salt/lib/python3.10/lib-dynload', '/opt/saltstack/salt/lib/python3.10/site-packages']
By debugging a bit more with the following file:
#!/bin/sh
"true" ''''
"export" "PATH=/opt/saltstack/salt/bin:/opt/saltstack/salt/extras-3.10/bin:$PATH:/opt/saltstack/salt/extras-3.10/bin:/opt/saltstack/salt/bin"
"echo" "$PATH"
#"echo" "Executing: $(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
#"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
"echo" "Executing: $(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
'''
# -*- coding: utf-8 -*-
import re
import sys
from salt.scripts import salt_pip
if __name__ == '__main__':
print(sys.path)
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(salt_pip())
I get now:
root@HOST:~# /opt/saltstack/salt/salt-pip install libvirt-python
/opt/saltstack/salt/bin:/opt/saltstack/salt/extras-3.10/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/saltstack/salt/extras-3.10/bin:/opt/saltstack/salt/bin
Executing: /opt/saltstack/salt/bin/python3.10 -m pip install libvirt-python
Collecting libvirt-python
Using cached libvirt-python-10.2.0.tar.gz (246 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: libvirt-python
Building wheel for libvirt-python (pyproject.toml) ... done
Created wheel for libvirt-python: filename=libvirt_python-10.2.0-cp310-cp310-linux_x86_64.whl size=502434 sha256=5b3dfea767aa72d38c5170a0a75c7a7c83393e7e2cab601f9f61bd0b6ff2446f
Stored in directory: /root/.cache/pip/wheels/5b/41/9b/8fc0e14c789b09a95b0db2581e139cf446881b782a54049d75
Successfully built libvirt-python
Installing collected packages: libvirt-python
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libglib-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgobject-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgio-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libxml2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libacl.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libaudit.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcap-ng.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgnutls.so.30 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnl-3.so.200 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnuma.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libselinux.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libapparmor.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libyajl.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh2.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libsasl2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtirpc.so.3 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpcre2-8.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libmount.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicuuc.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libp11-kit.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libidn2.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libunistring.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtasn1.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnettle.so.8 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libhogweed.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmp.so.10 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnghttp2.so.14 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/librtmp.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpsl.so.5 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libldap-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/liblber-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libzstd.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlidec.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libblkid.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicudata.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libstdc++.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcom_err.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlicommon.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libkeyutils.so.1 is not in /opt/saltstack/salt
Do not adjust rpath of /opt/saltstack/salt/lib/python3.10/site-packages/libvirtmod.cpython-310-x86_64-linux-gnu.so
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt-lxc.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libapparmor.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libselinux.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libglib-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgobject-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgio-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libxml2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libacl.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libaudit.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcap-ng.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgnutls.so.30 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnl-3.so.200 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnuma.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libyajl.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh2.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libsasl2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtirpc.so.3 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpcre2-8.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libmount.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicuuc.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libp11-kit.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libidn2.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libunistring.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtasn1.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnettle.so.8 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libhogweed.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmp.so.10 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnghttp2.so.14 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/librtmp.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpsl.so.5 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libldap-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/liblber-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libzstd.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlidec.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libblkid.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicudata.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libstdc++.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcom_err.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlicommon.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libkeyutils.so.1 is not in /opt/saltstack/salt
Do not adjust rpath of /opt/saltstack/salt/lib/python3.10/site-packages/libvirtmod_lxc.cpython-310-x86_64-linux-gnu.so
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt-qemu.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libvirt.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libglib-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgobject-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgio-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libxml2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libacl.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libaudit.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcap-ng.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgnutls.so.30 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnl-3.so.200 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnuma.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libselinux.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libapparmor.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libyajl.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh2.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libssh.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libsasl2.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtirpc.so.3 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpcre2-8.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libmount.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicuuc.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libp11-kit.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libidn2.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libunistring.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libtasn1.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnettle.so.8 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libhogweed.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libgmp.so.10 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libnghttp2.so.14 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/librtmp.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libpsl.so.5 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libldap-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/liblber-2.5.so.0 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libzstd.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlidec.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libblkid.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libicudata.so.72 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libstdc++.so.6 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libcom_err.so.2 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libbrotlicommon.so.1 is not in /opt/saltstack/salt
WARNING: In `rpath_only mode` but /lib/x86_64-linux-gnu/libkeyutils.so.1 is not in /opt/saltstack/salt
Do not adjust rpath of /opt/saltstack/salt/lib/python3.10/site-packages/libvirtmod_qemu.cpython-310-x86_64-linux-gnu.so
Successfully installed libvirt-python-10.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
So the TLDR is that salt-pip
needs to be modified as following:
# This is WRONG
# "exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
# This WORKS CORRECTLY
"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
After the modification is done, this will match what other reporters said about needing to call python3.10 -m pip:
Executing: /opt/saltstack/salt/bin/python3.10 -m pip install libvirt-python
If somebody wants to use a patch
file generated using diff
(I'm NOT an expert in using these tools 😞):
This salt-pip.patch
:
--- /opt/saltstack/salt/salt-pip 2024-04-30 10:23:30.040269798 +0200
+++ /opt/saltstack/salt/salt-pip 2024-04-30 10:28:57.879895204 +0200
@@ -1,6 +1,6 @@
#!/bin/sh
"true" ''''
-"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
+"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
'''
# -*- coding: utf-8 -*-
import re
Or salt-pip.patch
:
3c3
< "exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
---
> "exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
But patch --verbose --dry-run -d/ -p0 <salt-pip.patch
would complain but still work correctly ...
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- /opt/saltstack/salt/salt-pip 2024-04-30 10:23:30.040269798 +0200
|+++ /opt/saltstack/salt/salt-pip 2024-04-30 10:28:57.879895204 +0200
--------------------------
patching file /opt/saltstack/salt/salt-pip
Using Plan A...
Hunk #1 succeeded at 1.
done
So if I make a patch file for both /usr/bin/salt-pip
and /opt/saltstack/salt/salt-pip
which are effectively the same contents (but separate executables, i.e. NOT connected):
--- /opt/saltstack/salt/salt-pip 2024-04-30 10:23:30.040269798 +0200
+++ /opt/saltstack/salt/salt-pip 2024-04-30 10:28:57.879895204 +0200
@@ -1,6 +1,6 @@
#!/bin/sh
"true" ''''
-"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
+"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
'''
# -*- coding: utf-8 -*-
import re
--- /usr/bin/salt-pip 2024-04-30 10:23:30.040269798 +0200
+++ /usr/bin/salt-pip 2024-04-30 10:28:57.879895204 +0200
@@ -1,6 +1,6 @@
#!/bin/sh
"true" ''''
-"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" "$0" "$@"
+"exec" "$(dirname "$(readlink -f "$0")")/bin/python3.10" -m pip "$@"
'''
# -*- coding: utf-8 -*-
import re
Apply using:
patch --verbose -d/ -p0 <salt-pip.patch
This patches both files. Now salt-pip
(both of them) work correctly 😃 .
@nf-brentsaner, @timwsuqld, @DaAwesomeP , @max-arnold , @lburinov73 : Please see my (long) comment #65980 (comment) for a possible long-term fix.
The TLDR is a patch
of /usr/bin/salt-pip
and /opt/saltstack/salt/salt-pip
is needed to change $0
in the exec
line (which returns the full path to the file) to -m pip
(after which the command matches the workaround you all used).
It would be good if some of the Developers would implement this in the code though ...
EDIT 1: For completeness, here is the salt --versions-report
to show the Versions of salt
and its Dependencies that I am using:
root@HOST:~# salt --versions-report
Salt Version:
Salt: 3007.0
Python Version:
Python: 3.10.13 (main, Feb 19 2024, 03:31:20) [GCC 11.2.0]
Dependency Versions:
cffi: 1.16.0
cherrypy: unknown
dateutil: 2.8.2
docker-py: Not Installed
gitdb: 4.0.11
gitpython: 3.1.40
Jinja2: 3.1.3
libgit2: 1.5.0
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.7
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 23.1
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: 1.10.1
python-gnupg: 0.5.2
PyYAML: 6.0.1
PyZMQ: 25.1.2
relenv: 0.15.1
smmap: 5.0.1
timelib: 0.3.0
Tornado: 6.3.3
ZMQ: 4.3.4
Salt Package Information:
Package Type: onedir
System Versions:
dist: debian 12.5 bookworm
locale: utf-8
machine: x86_64
release: 6.1.0-20-amd64
system: Linux
version: Debian GNU/Linux 12.5 bookworm
The “workaround” of patching salt-pip
or directly calling python3 -m pip
is not a good one, because it causes the module to go into /opt/saltstack/salt/lib/python3.10/site-packages
, when it should go into /opt/saltstack/salt/extras-3.10
. When calling python3 -m pip
with --target=/opt/saltstack/salt/extras-3.10
, the problem reappears, so this is not a solution either.
I investigated the issue more closely and found that actually, there are two bugs in play here:
The first one only affects Salt 3006.6 and is rooted in the version of relenv that is distributed with that version of Salt. This bug causes the None
values to appear in sitecustomize.py
and has been fixed in relenv 0.15.0, so it only affected Salt 3006.6 and is fixed in Salt 3006.7 and newer versions.
This bug caused the TypeError: expected str, bytes or os.PathLike object, not NoneType
message.
However, installing mysqlclient
still fails in Salt 3006.8, but now with the AssertionError
(BTW, I believe that #66311 is caused by exactly the same bug). The workaround for fixing the AssertionError
is removing the assert not path in sys.path
line in /opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/build_env.py
. Unfortunately, this leads to yet another problem where the setuptools
module cannot be found.
The underyling problem is that the relenv
module somehow messes up the build environment for pip, as can be proven by applying the following workaround (do not try this on a production system – it is very likely to mess up other things):
Remove relenv (rm -r /opt/saltstack/salt/lib/python3.10/site-packages/relenv*
) and fix /opt/saltstack/salt/lib/python3.10/site-packages/_sysconfigdata__linux_x86_64-linux-gnu.py
by replacing the line
toolchain = DATA_DIR / "toolchain" / get_triplet()
with
toolchain = Path("/usr")
The latter part is needed because with relenv
being absent, GCC will not be found without this change.
Now, the mysqlclient
module can be installed with /opt/saltstack/salt/salt-pip install mysqlclient
and ends up in /opt/saltstack/salt/extras-3.10
as expected.
So, the underyling issue definitely is in relenv
. Unfortunately, relenv
messes around with a lot of Python internals, so it is not easy to find the code that is actually causing this problem.
I just realized that the description of my workaround was both incomplete and more invasive than it had to be. Here is the correct description:
First, edit /opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py
and remove or comment out the last line:
sys.meta_path = [importer] + sys.meta_path
Second, edit /opt/saltstack/salt/lib/python3.10/_sysconfigdata__linux_x86_64-linux-gnu.py
and replace
toolchain = DATA_DIR / "toolchain" / get_triplet()
with
toolchain = pathlib.Path("/usr")
That’s it. Now salt-pip install mysqlclient
works as expected.
…and I found an even better, simpler workaround: In Salt 3006.8 (I haven’t tested other versions), edit /opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py
and comment out line 786, which should look like this:
Wrapper("pip._internal.commands.install", wrap_cmd_install),
This fixes the problem without any other changes and brings us much closer to finding the code that is actually causing the whole issue.
I think I finally found the underlying issue and have opened an issue in the relenv project: saltstack/relenv#182
There, I also provide an alternative workaround.
I think I finally found the underlying issue and have opened an issue in the relenv project: saltstack/relenv#182
There, I also provide an alternative workaround.
Glad you found a better Solution / Fix than my Workaround 👍.
My Workaround might not have been optimal for sure, but it's better than being stuck waiting for the next release which might happen in 6 months or so 😞.
And from what I could test, (my workaround) seemed to work.
I agree though ... long Term your Solution is probably better and that's the way forward 👍.
@smarsching : The issue (with salt-pip
) also occurs on Salt 3007.0 (tested on Debian Bookworm 12 AMD64), so definitively NOT only relevant to 3006.X Versions of Salt.
smarsching Thanks for your deep dive on this!
This should be resolved in the next 3006.x release.
What about the released 3007.1 or the upcoming 3007.2 (?) ?
What about the released 3007.1 or the upcoming 3007.2 (?) ?
Yes, it will be in 3007.2
as well as 3006.9
. Closing.
EDIT: Installing the following system packages on Debian fixed the issue here:
sudo apt-get install pkg-config python3-dev default-libmysqlclient-dev build-essential
This just happened here with 3006.9
(onedir installation) on Debian bullseye when running:
salt-pip install mysqlclient
salt-call pip.list
local:
----------
CherryPy:
18.6.1
Jinja2:
3.1.4
MarkupSafe:
2.1.2
PyYAML:
6.0.1
certifi:
2024.7.4
cffi:
1.14.6
charset-normalizer:
3.2.0
cheroot:
8.5.2
contextvars:
2.4
croniter:
2.0.5
cryptography:
42.0.5
distro:
1.5.0
idna:
3.7
immutables:
0.15
importlib-metadata:
6.0.0
jaraco.classes:
3.2.1
jaraco.collections:
3.4.0
jaraco.functools:
2.0
jaraco.text:
3.5.1
jmespath:
1.0.1
looseversion:
1.0.2
more-itertools:
5.0.0
msgpack:
1.0.2
packaging:
22.0
pip:
23.3.2
portend:
2.4
psutil:
5.8.0
pyOpenSSL:
24.0.0
pycparser:
2.21
pycryptodomex:
3.19.1
python-dateutil:
2.8.1
python-gnupg:
0.4.8
pytz:
2022.1
pyzmq:
23.2.0
relenv:
0.17.0
requests:
2.32.3
rpm-vercmp:
0.1.2
salt:
3006.9
setproctitle:
1.3.2
setuptools:
68.2.2
six:
1.16.0
tempora:
4.1.1
timelib:
0.2.5
urllib3:
1.26.18
wheel:
0.43.0
zc.lockfile:
1.4
zipp:
3.6.0