ilastik/publish-conda-stack

Remove libgcc dependency for all recipes

stuarteberg opened this issue · 5 comments

(This isn't an issue for this repo per se, but this repo seems like the right place for us to track build-related issues. Let me know if you have a better suggestion.)

For Mac, we have already removed our dependency on conda's libgcc package from all of our recipes. But on Linux, the dependency remains. Now that we use a docker container for our builds, which already comes with a suitable libgcc of its own (and we don't intend to support any users with older OSes than that), we can stop using conda's libgcc package and just assume that our users' own machines already have a new-enough libgcc pre-installed.

You can easily see which packages currently depend on libgcc by attempting to remove libgcc from your environment:

$ conda remove -n ilastik-env libgcc
Fetching package metadata ...........
Solving package specifications: .

Package plan for package removal in environment /opt/conda/envs/ilastik-env:

The following packages will be REMOVED:

    dpct:                            1.2.post11-py36_0_ga319138        ilastik-forge
    hytra:                           1.1.4.post9-py36_0_g756bbff       ilastik-forge
    iiboost:                         0.2.post10-36_0_g08c3d66          ilastik-forge
    ilastik-dependencies-no-solvers: 1.3-h4ad2b3e_2                    ilastik-forge
    ilastikrag:                      0.1.3.post3-py36_0_g0fa508f       ilastik-forge
    ilastiktools:                    0.2.post5-np112py36_0_gdfe578c    ilastik-forge
    lemon:                           1.2.4-hda0c028_3                  ilastik-forge
    libgcc:                          4.8.5-2                           defaults
    vigra:                           1.11.1.post8-np112py36_5_gfb42744 ilastik-forge
    wsdt:                            0.2.post16-py36_0_g76f938f        ilastik-forge

Edit: Of course, some of those don't depend on libgcc directly, but inherit the dependency by transitivity. The list of recipes that need to be changed is:

  • dpct
  • iiboost
  • ilastiktools
  • lemon
  • vigra

Just a progress update:

So, I have removed all dependencies on libgcc and the conda-forge provided gcc from all packages we build ourselves in order to be able to build with a newer compiler. This was necessary in order to build a current version of nifty (which we need for the improved carving workflow). For building I have used the conda linux-anvil with the developer toolset v4 (gcc 5.3.1). For the libs we compile ourselves, I have enforced the "old-style-abi" with the -D_GLIBCXX_USE_CXX11_ABI=0 cxx flag.

I tested ilastik with this new environment a bit and haven't encountered any crashes so far. But well. Whoever wants to give it a go can create an environment with

conda create -n ilastik-newgcc -c kdominik -c conda-forge ilastik-dependencies-no-solvers

The main problem that I see with moving away from the conda-forge gcc is, that if any conda-forge package uses pybind11 for python bindings, it will clash with ours. Apparently you get segfaults when using pybind11-based python packages built with different compiler versions.

I wonder if we could include testing for this into our pre-release binary testing? I know it could be any package, but there are not so many we call through pybind11.

help("modules")

this segfaults it for me if I have such modules present.

Good, then let's include it somehow into the pre-release binary tests.

done