conda's libgcc package masks the system libstdc++.so

Question

conda's libgcc package masks the system libstdc++.so

stuarteberg opened this issue 8 years ago · 1 comments

[I'm copying this old issue from the conda-recipes issue tracker because that tracker is being disabled. See also: https://github.com/ilastik/ilastik/issues/1227]

(This issue applies to Linux only, I think.**) This issue has been discussed previously on the conda mailing list:

When a package requires libgcc, it will download (among others) libstdc++.so. That version of libstdc++.so is somewhat old. When it appears on the loader RPATH, it masks the system's libstdc++.so, which might contain newer symbols required by third-party packages used by the OS (e.g. OpenGL).

Potential solutions to this problem include the following:

Add a post-link.sh script to the libgcc package that checks the user's system for a newer version of libstdc++.so. If a newer version exists, it should be backwards-compatible with any packages that we built against conda's version libstdc++.so, and therefore there's no need to install conda's copy of libstdc++.so at all.
Upgrade the libgcc package to a cutting-edge version to avoid this problem (and maybe gcc, too). To avoid this problem in the future, it will need to be updated regularly, and packages that depend on libgcc should avoid requiring a specific version.
As wojdyr described in thread 1 above, something called devtoolset-2 might be an alternative solution. seibert mentioned that that's what the numba developers use. ~~IIUC, it would require upgrading Anaconda's build VM from CentOS-5 to CentOS-6, which would change Anaconda's minimum system requirements on Linux (for C++ packages, at least).~~ [Edit: Not if we restrict ourselves to devtoolset v2. See wojdyr's comment below.]

Option 1 seems most attractive to me. (In thread 2 above, manually uninstalling the libgcc package solved the user's problem.)

**I think OSX is not affected because Mach-O dylibs reference libstdc++.dylib via an absolute path, so conda's copy shouldn't conflict with the system copy. And even if it did conflict, OSX ships a really old version of libstdc++ anyway, so our version will always be better.

wojdyr commented:

Re 3.
devtoolset v2 works on RHEL5 and is available also in RHEL clones:
http://linux.web.cern.ch/linux/devtoolset/
http://people.centos.org/tru/devtools-2/
(I'm using the latter).
It was released 2 years ago and has gcc 4.8.2.
The later version (v3 and now v4 is in preparation or maybe already released) are only RHEL6+.

wojdyr commented:

Actually I don't see libgcc in requirements of any anaconda package -- I'm looking at http://repo.continuum.io/pkgs/free/linux-64/repodata.json
Possibly it's required by packages in other repositories? Or it's required but not listed explicitely.

stuarteberg commented:

Actually I don't see libgcc in requirements of any anaconda package

Yeah, the good news is that this isn't a terribly urgent issue, at least for python users.

So far, most python packages can be built with the default compiler on CentOS-5, so they don't need to depend on conda's gcc-4.8 package (or libgcc). For instance, few, if any, require C++, much less a new-ish C++ compiler.

But in the future it would be nice if conda can cleanly support python packages that depend on modern C++.

Possibly it's required by packages in other repositories?

Yes, apparently R packages need it: Try conda create -n testenv -c r r-essentials. (It will download libgcc.) That's how the user in thread 2 discovered this problem.

msarahan commented:

Sorry about the delay here. We are working on modernizing our build stack to GCC 5.2. You are more than welcome to try it out: https://hub.docker.com/r/msarahan/conda_builder_linux/ and ContinuumIO/docker-images#20/

Note that GCC compiles with the GCC 4 ABI compatibility mode.

Concomitant with this, we'll also be updating the libgcc package for the new libgcc and libstdc++. These will become standard packages on Linux with the new compiler, I think.

This is not my area of specialty, so if I'm doing anything daft, please point me in the right direction.

wojdyr commented:

msarahan: it doesn't solve the problem from this issue in general, but it may work in all practical cases for now. It should work unless the system GCC is newer than the one from conda AND you load a system library that uses newly changed functions from libstdc++.

msarahan commented:

Thanks. I'm not sure there is a truly general solution. I will build a new libgcc package with the libraries from gcc 5.2 today. For testing purposes, I will put it on my anaconda.org account for now. I'll post here when I have that up, and will appreciate test feedback. Also, I would value your input on whether we should only provide the compiler in a docker image (to ensure glibc maximum compatibility) or if making a conda package of the compiler adds enough convenience that potential glibc problems are worth ignoring. How much overhead would docker be for you?

wojdyr commented:

I can speak only for myself, but I'm fine with using docker. I've never used it before, but I'll need to get familiar with it anyway, sooner or later.
As a potential user of packages from Anaconda cloud, I'm actually bothered that I don't know what distro they were compiled on and I'm not sure if the packages will work on my system. So I'd be happy if everyone was using the same docker image.

stuarteberg commented:

msarahan: it doesn't solve the problem from this issue in general, but it may work in all practical cases for now. It should work unless the system GCC is newer than the one from conda AND you load a system library that uses newly changed functions from libstdc++

Agreed. By upgrading to gcc-5, you are effectively choosing option 2 from the issue description above. That will work in the near term, but as noted, we'll need to upgrade the gcc package frequently to stay ahead of OS distros.

I'm actually bothered that I don't know what distro they were compiled on

From what I understand, the packages are compiled on CentOS 5.11. For my own packages, I build on a CentOS 5.11 VM, and it seems to be compatible with everything in Anaconda. If you're interested, I could provide a link to my VM image.

msarahan commented:

Anaconda is compiled on CentOS 5. But generally, for anaconda.org, there is no indication what any arbitrary package from any user is compiled with.

Our (in-testing) docker image is at:https://hub.docker.com/r/msarahan/conda_builder_linux/

An updated libgcc is available on my channel:

conda install -c msarahan libgcc

This is compiled with GCC 4 compatibility.

insertinterestingnamehere commented:

This new toolchain is great. I've uploaded experimental builds of libdynd and dynd-python (both require C++14) and they appear to be working well everywhere I've tested them. They are available via conda install dynd-python -c iandh -c msarahan.

Answer 1 · 2016-12-13T19:45:31.000Z

We now use a special build of the libgcc and gcc packages that include a custom post-link.sh script:

https://github.com/ilastik/ilastik-build-conda/blob/master/libgcc/remove-obsolete-libstdcxx.sh

That script inspects the system's libstdc++.so for newer symbols than the ones offered by our (ilastik's) libstdc++.so. If the system's appears to be newer, it deletes our libstdc++.so file (and a few others).

That hack seems to suffice for now.