OpenEXR (python): libstdc++.so.6: version `GLIBCXX_3.4.29' not found
kunaltyagi opened this issue · 8 comments
Conda-forge documentation
- I could not solve my problem using the conda-forge documentation.
Installed packages
Name Version Build Channel
──────────────────────────────────────────────────────────────────────────
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
aom 3.6.1 h59595ed_0 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
cairo 1.18.0 h3faef2a_0 conda-forge
dav1d 1.2.1 hd590300_0 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
ffmpeg 6.0.0 gpl_h334edf3_105 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
fribidi 1.0.10 h36c2ea0_0 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
gmp 6.2.1 h58526e2_0 conda-forge
gnutls 3.7.8 hf3e180e_0 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
harfbuzz 8.2.1 h3d44ed6_0 conda-forge
icu 73.2 h59595ed_0 conda-forge
imath 3.1.9 hfc55251_0 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
libass 0.17.1 h8fe9dca_1 conda-forge
libdrm 2.4.114 h166bdaf_0 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc 5.2.0 0 conda-forge
libgcc-ng 13.2.0 h807b86a_2 conda-forge
libglib 2.78.0 hebfc3b9_0 conda-forge
libgomp 13.2.0 h807b86a_2 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libidn2 2.3.4 h166bdaf_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpciaccess 0.17 h166bdaf_0 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libsqlite 3.43.2 h2797004_0 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_2 conda-forge
libtasn1 4.19.0 h166bdaf_0 conda-forge
libunistring 0.9.10 h7f98852_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libva 2.20.0 hd590300_0 conda-forge
libvpx 1.13.1 h59595ed_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxml2 2.11.5 h232c23b_1 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
ncurses 6.4 hcb278e6_0 conda-forge
nettle 3.8.1 hc379101_1 conda-forge
openexr 3.2.1 h3f0fd8d_0 conda-forge
openexr-python 1.3.9 py310h905be85_3 conda-forge
openh264 2.3.1 hcb278e6_2 conda-forge
openssl 3.1.3 hd590300_0 conda-forge
p11-kit 0.24.1 hc5aa10d_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
pip 23.3.1 pyhd8ed1ab_0 conda-forge
pixman 0.42.2 h59595ed_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
python 3.10.12 hd12c33a_0_cpython conda-forge
python_abi 3.10 4_cp310 conda-forge
readline 8.2 h8228510_1 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
svt-av1 1.7.0 h59595ed_0 conda-forge
tk 8.6.13 h2797004_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
wheel 0.41.2 pyhd8ed1ab_0 conda-forge
x264 1!164.3095 h166bdaf_2 conda-forge
x265 3.5 h924138e_3 conda-forge
xorg-fixesproto 5.0 h7f98852_1002 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.7 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forgeEnvironment info
libmamba version : 1.5.1
micromamba version : 1.5.1
curl version : libcurl/7.88.1 OpenSSL/3.1.2 zlib/1.2.13 zstd/1.5.5 libssh2/1.11.0 nghttp2/1.52.0
libarchive version : libarchive 3.6.2 zlib/1.2.13 bz2lib/1.0.8 libzstd/1.5.2
envs directories : /opt/micromamba/envs
package cache : /opt/micromamba/pkgs
~/.mamba/pkgs
environment : dummy (active)
env location : /opt/micromamba/envs/dummy
user config files : ~/.mambarc
populated config files :
virtual packages : __unix=0=0
__linux=5.4.0=0
__glibc=2.31=0
__archspec=1=x86_64
channels :
base environment : /opt/micromamba
platform : linux-64Issue
channels:
- conda-forge
- defaults
dependencies:
- python=3.10
# - libgcc=5.2.0
- openexr=3.2.1
- openexr-python=1.3.9
Using these dependencies, and import OpenEXR in python, I get the following error:
ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /opt/micromamba/envs/dummy/lib/libOpenEXR.so.31)
The libstdc++ installed is not taken from conda. This seems very weird to me. Also, installing libgcc doesn't resolve the issue
I cannot recreate this:
$ docker run --rm -it mambaorg/micromamba:latest
## docker session
## create environment
(base) mambauser@64332bdaa1ba:/tmp$ micromamba create -qy \
-c conda-forge -n foo \
python=3.10 openexr=3.2.1 openexr-python=1.3.9
## check environment result
(base) mambauser@64332bdaa1ba:/tmp$ micromamba activate foo
(foo) mambauser@64332bdaa1ba:/tmp$ micromamba list -n foo
List of packages in environment: "/opt/conda/envs/foo"
Name Version Build Channel
────────────────────────────────────────────────────────────────
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
imath 3.1.9 hfc55251_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 13.2.0 h807b86a_2 conda-forge
libgomp 13.2.0 h807b86a_2 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libsqlite 3.43.2 h2797004_0 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_2 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
ncurses 6.4 hcb278e6_0 conda-forge
openexr 3.2.1 h3f0fd8d_0 conda-forge
openexr-python 1.3.9 py310h905be85_3 conda-forge
openssl 3.1.3 hd590300_0 conda-forge
pip 23.3.1 pyhd8ed1ab_0 conda-forge
python 3.10.12 hd12c33a_0_cpython conda-forge
python_abi 3.10 4_cp310 conda-forge
readline 8.2 h8228510_1 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
tk 8.6.13 h2797004_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
wheel 0.41.2 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
## try loading
(foo) mambauser@64332bdaa1ba:/tmp$ python
Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import OpenEXR
>>>
## no errorIt is strange that the environment here is far more minimal than OP reports. Perhaps try recreating the environment from scratch again.
Have you verified that there is no user site with the OpenEXR module installed? See https://stackoverflow.com/a/70961159/570918.
Thanks for the test. I found that the issue was some other packages using pip with --extra-index-url
Thanks for showing how to use docker to narrow down the issue, lovely idea
I narrowed it down even more. This is really weird:
python -c "import OpenEXR; import torch" # works without issue
python -c "import torch; import OpenEXR" # raises the same errorThis happen only with specific version of torch:
- issue only with
--extra-index-url https://download.pytorch.org/whl/rocm5.4.2 torch==2.0.1+rocm5.4.2 torch==2.0.1has no issues
I'll report it to the torch team. But posting here in case someone comes looking for it in future (most likely myself)
My guess is OpenEXR is statically linking or vendoring libstdc++ in their wheels. The result being when they are imported first, the library loader resolves these GLIBCXX symbols. So by the time torch is loaded, it just reuses the already loaded symbols
TBH would be more worried about OpenEXR doing something non-standard in their wheels that is hidden by the fact it seems to work
Edit: It is possible the PyTorch wheels also have some issues in specific binaries. Can't comment on that though
Can we check the openexr-python conda recipe? I don't know how the binaries are generated
Thought you were saying both were pip installed. Is that not the case?
No, the openexr are via conda (openexr=3.2.1 openexr-python=1.3.9) and torch is via pip
Ok then I don't think what I said above applies to the Conda package
The Conda package for openexr would have dynamically linked to libstdc++, which would have been pulled in as a Conda dependency. torch would then be reusing the symbols already loaded by openexr
When importing torch first, it would look for libstdc++ at the system level and not find it
Would continue with raising a PyTorch issue. It is possible that they say the OS where torch was installed is not supported, but the fact that some wheels work and some don't may mean it was a one off build issue