Installing tsnecuda and faiss-gpu breaks conda
Closed this issue · 2 comments
samuelbroscheit commented
Hi,
just installing tsnecuda with
conda install tsnecuda cuda100 -c cannylab
and trying to do a TSNE.fit() did yield
OSError: libfaiss.so: cannot open shared object file: No such file or directory
So I tried to install faiss-gpu which results in a dependency error. Steps to recreate:
conda create --name cuda python=3.7
conda activate cuda
conda install faiss-gpu cudatoolkit=10.0 -c pytorch
conda install tsnecuda cuda100 -c cannylab
which results in
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Package blas conflicts for:
mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> blas[version='*|1.0',build=openblas]
mkl_fft -> blas[version='*|1.0',build=mkl]
mkl-service -> numpy[version='>=1.11.3,<2.0a0'] -> blas=[build=mkl]
faiss-gpu -> blas=[build=mkl]
faiss-gpu -> numpy[version='>=1.11'] -> blas==1.0=mkl
mkl_random -> blas[version='*|1.0',build=mkl]
tsnecuda -> openblas -> blas[version='*|1.0',build=openblas]
numpy -> blas[version='*|*|1.0',build='openblas|mkl|openblas|mkl']
mkl-service -> numpy[version='>=1.11.3,<2.0a0'] -> numpy-base==1.11.3=py37h3dfced4_9 -> blas[version='*|1.0',build=openblas]
blas
mkl-service -> blas==1.0=mkl
numpy-base -> blas[version='*|*|1.0',build='openblas|mkl|openblas|mkl']
mkl_fft -> numpy-base[version='>=1.0.6,<2.0a0'] -> blas[version='*|1.0',build=openblas]
Package numpy-base conflicts for:
mkl-service -> numpy[version='>=1.11.3,<2.0a0'] -> numpy-base[version='1.11.3|1.14.3|1.14.3|1.14.3|1.14.4|1.14.4|1.14.4|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.6|1.14.6|1.14.6|1.14.6|1.15.0|1.15.0|1.15.0|1.15.0|1.15.1|1.15.1|1.15.1|1.15.1',build='py37h74e8950_0|py35h7cdd4dd_0|py37h2f8d375_4|py36h2f8d375_4|py35h2f8d375_4|py37h2b20989_4|py37h2b20989_3|py37h2b20989_2|py36h2b20989_4|py36h2b20989_3|py36h2b20989_2|py36h2b20989_0|py27h2b20989_4|py27h2b20989_3|py27h2b20989_1|py27h2b20989_0|py36h2b20989_0|py35h2b20989_0|py27h2b20989_0|py27h0ea5e3f_1|py37hde5b4d6_11|py37hdbf6ddf_7|py37h3dfced4_9|py36hde5b4d6_12|py36hde5b4d6_11|py36hdbf6ddf_7|py36h81de0dd_10|py35hdbf6ddf_8|py35h81de0dd_9|py35h81de0dd_10|py35h3dfced4_9|py27hde5b4d6_11|py27h81de0dd_9|py27h81de0dd_10|py27h3dfced4_9|py27hdbf6ddf_7|py27hdbf6ddf_8|py27hde5b4d6_12|py36h3dfced4_9|py36h81de0dd_9|py36hdbf6ddf_8|py37h81de0dd_10|py37h81de0dd_9|py37hdbf6ddf_8|py37hde5b4d6_12|py35h0ea5e3f_1|py36h0ea5e3f_1|py27h2b20989_2|py35h2b20989_4|py36h2b20989_1|py37h2b20989_1|py27h2f8d375_4|py27h7cdd4dd_0|py36h7cdd4dd_0|py37h7cdd4dd_0|py27h74e8950_0|py35h74e8950_0|py36h74e8950_0']
mkl_fft -> numpy[version='>=1.11.3,<2.0a0'] -> numpy-base[version='1.11.3|1.14.3|1.14.3|1.14.3|1.14.4|1.14.4|1.14.4|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.6|1.14.6|1.14.6|1.14.6|1.15.0|1.15.0|1.15.0|1.15.0|1.15.1|1.15.1|1.15.1|1.15.1|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3',build='py37hdbf6ddf_6|py36hdbf6ddf_7|py36hdbf6ddf_6|py27hdbf6ddf_7|py27hdbf6ddf_6|py37h74e8950_0|py35h7cdd4dd_0|py37h2f8d375_4|py36h2f8d375_4|py35h2f8d375_4|py37h2b20989_4|py37h2b20989_3|py37h2b20989_2|py36h2b20989_4|py36h2b20989_3|py36h2b20989_2|py36h2b20989_0|py27h2b20989_4|py27h2b20989_3|py27h2b20989_1|py27h2b20989_0|py36h2b20989_0|py35h2b20989_0|py27h2b20989_0|py27h0ea5e3f_1|py37hde5b4d6_11|py37hdbf6ddf_7|py37h3dfced4_9|py36hde5b4d6_12|py36hde5b4d6_11|py36hdbf6ddf_7|py36h81de0dd_10|py35hdbf6ddf_8|py35h81de0dd_9|py35h81de0dd_10|py35h3dfced4_9|py27hde5b4d6_11|py27h81de0dd_9|py27h81de0dd_10|py27h3dfced4_9|py27hdbf6ddf_7|py27hdbf6ddf_8|py27hde5b4d6_12|py36h3dfced4_9|py36h81de0dd_9|py36hdbf6ddf_8|py37h81de0dd_10|py37h81de0dd_9|py37hdbf6ddf_8|py37hde5b4d6_12|py35h0ea5e3f_1|py36h0ea5e3f_1|py27h2b20989_2|py35h2b20989_4|py36h2b20989_1|py37h2b20989_1|py27h2f8d375_4|py27h7cdd4dd_0|py36h7cdd4dd_0|py37h7cdd4dd_0|py27h74e8950_0|py35h74e8950_0|py36h74e8950_0|py35hdbf6ddf_7|py37hdbf6ddf_7']
numpy -> numpy-base[version='1.11.3|1.14.3|1.14.3|1.14.3|1.14.3|1.14.3|1.14.3|1.14.4|1.14.4|1.14.4|1.14.4|1.14.4|1.14.4|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.3|1.15.3|1.15.3|1.15.3|1.15.3|1.15.3|1.15.4|1.15.4|1.15.4|1.15.4|1.15.4|1.15.4|1.15.4|1.15.4|1.15.4|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.2|1.16.2|1.16.2|1.16.2|1.16.2|1.16.2|1.16.3|1.16.3|1.16.3|1.16.3|1.16.3|1.16.3|1.16.4|1.16.4|1.16.4|1.16.4|1.16.4|1.16.4|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|>=1.9.3,<2.0a0',build='py37hdbf6ddf_6|py37h2b20989_7|py36hdbf6ddf_7|py36hdbf6ddf_6|py36h2b20989_6|py27hdbf6ddf_7|py27hdbf6ddf_6|py36hde5b4d6_0|py36h2f8d375_0|py27hde5b4d6_0|py36h2f8d375_0|py36h2f8d375_0|py27hde5b4d6_0|py37hde5b4d6_1|py37h2f8d375_1|py37h2f8d375_0|py36hde5b4d6_1|py36hde5b4d6_0|py36h2f8d375_1|py36h2f8d375_0|py27hde5b4d6_0|py27h2f8d375_0|py37hde5b4d6_1|py37h2f8d375_1|py37h2f8d375_0|py36hde5b4d6_1|py36hde5b4d6_0|py36h2f8d375_1|py36h2f8d375_0|py27h2f8d375_0|py37hde5b4d6_0|py37h81de0dd_0|py36hde5b4d6_0|py36h2f8d375_0|py27hde5b4d6_0|py27h81de0dd_0|py37h81de0dd_0|py36h2f8d375_0|py27h81de0dd_0|py37h81de0dd_0|py37h2f8d375_1|py36h81de0dd_0|py36h2f8d375_0|py27h81de0dd_0|py36h81de0dd_0|py35h81de0dd_0|py35h74e8950_0|py36h7cdd4dd_0|py35h7cdd4dd_0|py37h81de0dd_4|py36h2f8d375_4|py35h81de0dd_4|py35h2f8d375_4|py37hdbf6ddf_4|py37hdbf6ddf_3|py37hdbf6ddf_2|py37hdbf6ddf_1|py37h2b20989_3|py36hdbf6ddf_4|py36hdbf6ddf_2|py36hdbf6ddf_1|py36hdbf6ddf_0|py36h2b20989_4|py36h2b20989_3|py36h2b20989_2|py36h2b20989_0|py35hdbf6ddf_4|py27hdbf6ddf_4|py27hdbf6ddf_3|py27h2b20989_4|py27h2b20989_3|py36hdbf6ddf_0|py36h2b20989_0|py35hdbf6ddf_0|py27h2b20989_0|py35h9be14a7_1|py27h0ea5e3f_1|py37hdbf6ddf_8|py37hdbf6ddf_7|py37h81de0dd_10|py37h7cdd4dd_9|py37h3dfced4_9|py37h2f8d375_12|py37h2f8d375_10|py37h2b20989_7|py36hde5b4d6_12|py36hde5b4d6_11|py36hdbf6ddf_7|py36h81de0dd_10|py36h7cdd4dd_9|py36h74e8950_10|py36h3dfced4_9|py36h2f8d375_11|py36h2f8d375_10|py35hdbf6ddf_8|py35h81de0dd_10|py35h7cdd4dd_9|py35h74e8950_10|py35h3dfced4_9|py35h2f8d375_10|py27hde5b4d6_11|py27hdbf6ddf_7|py27h81de0dd_9|py27h81de0dd_10|py27h7cdd4dd_9|py27h74e8950_9|py27h74e8950_10|py27h3dfced4_9|py27h2f8d375_12|py27h2f8d375_10|py27h2b20989_8|py27h2b20989_7|py27h2f8d375_11|py27hdbf6ddf_8|py27hde5b4d6_12|py35h2b20989_8|py35h74e8950_9|py35h81de0dd_9|py36h2b20989_7|py36h2b20989_8|py36h2f8d375_12|py36h74e8950_9|py36h81de0dd_9|py36hdbf6ddf_8|py37h2b20989_8|py37h2f8d375_11|py37h74e8950_10|py37h74e8950_9|py37h81de0dd_9|py37hde5b4d6_11|py37hde5b4d6_12|py27h9be14a7_1|py35h0ea5e3f_1|py36h0ea5e3f_1|py36h9be14a7_1|py27hdbf6ddf_0|py35h2b20989_0|py27h2b20989_0|py27h2b20989_1|py27h2b20989_2|py27hdbf6ddf_0|py27hdbf6ddf_1|py27hdbf6ddf_2|py35h2b20989_4|py35hdbf6ddf_0|py36h2b20989_1|py36hdbf6ddf_3|py37h2b20989_1|py37h2b20989_2|py37h2b20989_4|py27h2f8d375_4|py27h81de0dd_4|py36h81de0dd_4|py37h2f8d375_4|py27h3dfced4_0|py27h7cdd4dd_0|py35h3dfced4_0|py36h3dfced4_0|py37h3dfced4_0|py37h7cdd4dd_0|py27h2f8d375_0|py27h74e8950_0|py27h81de0dd_0|py35h2f8d375_0|py36h2f8d375_0|py36h74e8950_0|py37h2f8d375_0|py37h74e8950_0|py37h81de0dd_0|py27h2f8d375_0|py27h2f8d375_1|py27h81de0dd_1|py35h2f8d375_0|py35h81de0dd_0|py36h2f8d375_1|py36h81de0dd_1|py37h2f8d375_0|py37h81de0dd_1|py27h2f8d375_0|py36h81de0dd_0|py37h2f8d375_0|py27h2f8d375_0|py36h81de0dd_0|py37h2f8d375_0|py27h2f8d375_1|py27hde5b4d6_0|py27hde5b4d6_1|py37hde5b4d6_0|py27h2f8d375_1|py27hde5b4d6_1|py37hde5b4d6_0|py27h2f8d375_0|py36hde5b4d6_0|py37h2f8d375_0|py37hde5b4d6_0|py27h2f8d375_0|py27hde5b4d6_0|py36hde5b4d6_0|py37h2f8d375_0|py37hde5b4d6_0|py27h2f8d375_0|py37h2f8d375_0|py37hde5b4d6_0|py27h2b20989_6|py27h2b20989_7|py35h2b20989_7|py35hdbf6ddf_7|py36h2b20989_7|py37h2b20989_6|py37hdbf6ddf_7']
mkl_fft -> numpy-base[version='>=1.0.6,<2.0a0']
faiss-gpu -> numpy[version='>=1.11'] -> numpy-base[version='1.11.3|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.6|1.14.6|1.15.0|1.15.0|1.15.1|1.15.1|1.15.2|1.15.2|1.15.3|1.15.4|1.15.4|1.16.0|1.16.0|1.16.1|1.16.1|1.16.2|1.16.3|1.16.4',build='py37hde5b4d6_0|py37hde5b4d6_1|py37hde5b4d6_1|py37hde5b4d6_0|py37hde5b4d6_0|py37h81de0dd_0|py37h81de0dd_0|py37h74e8950_0|py37h81de0dd_4|py37h2f8d375_4|py37hdbf6ddf_4|py37hdbf6ddf_1|py37h2b20989_4|py37h2b20989_2|py37h2b20989_1|py37hde5b4d6_12|py37hdbf6ddf_8|py37hdbf6ddf_7|py37h81de0dd_10|py37h2b20989_7|py37h2b20989_8|py37h3dfced4_9|py37h74e8950_9|py37h7cdd4dd_9|py37h81de0dd_9|py37hde5b4d6_11|py37h2b20989_3|py37hdbf6ddf_2|py37hdbf6ddf_3|py37h3dfced4_0|py37h7cdd4dd_0|py37h81de0dd_1|py37h81de0dd_0|py37h81de0dd_0|py37hde5b4d6_0|py37hde5b4d6_0|py37hde5b4d6_0']
mkl_random -> numpy[version='>=1.11.3,<2.0a0'] -> numpy-base[version='1.11.3|1.14.3|1.14.3|1.14.3|1.14.4|1.14.4|1.14.4|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.6|1.14.6|1.14.6|1.14.6|1.15.0|1.15.0|1.15.0|1.15.0|1.15.1|1.15.1|1.15.1|1.15.1|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3|1.9.3',build='py37hdbf6ddf_6|py36hdbf6ddf_7|py36hdbf6ddf_6|py27hdbf6ddf_7|py27hdbf6ddf_6|py37h74e8950_0|py35h7cdd4dd_0|py37h2f8d375_4|py36h2f8d375_4|py35h2f8d375_4|py37h2b20989_4|py37h2b20989_3|py37h2b20989_2|py36h2b20989_4|py36h2b20989_3|py36h2b20989_2|py36h2b20989_0|py27h2b20989_4|py27h2b20989_3|py27h2b20989_1|py27h2b20989_0|py36h2b20989_0|py35h2b20989_0|py27h2b20989_0|py27h0ea5e3f_1|py37hde5b4d6_11|py37hdbf6ddf_7|py37h3dfced4_9|py36hde5b4d6_12|py36hde5b4d6_11|py36hdbf6ddf_7|py36h81de0dd_10|py35hdbf6ddf_8|py35h81de0dd_9|py35h81de0dd_10|py35h3dfced4_9|py27hde5b4d6_11|py27h81de0dd_9|py27h81de0dd_10|py27h3dfced4_9|py27hdbf6ddf_7|py27hdbf6ddf_8|py27hde5b4d6_12|py36h3dfced4_9|py36h81de0dd_9|py36hdbf6ddf_8|py37h81de0dd_10|py37h81de0dd_9|py37hdbf6ddf_8|py37hde5b4d6_12|py35h0ea5e3f_1|py36h0ea5e3f_1|py27h2b20989_2|py35h2b20989_4|py36h2b20989_1|py37h2b20989_1|py27h2f8d375_4|py27h7cdd4dd_0|py36h7cdd4dd_0|py37h7cdd4dd_0|py27h74e8950_0|py35h74e8950_0|py36h74e8950_0|py35hdbf6ddf_7|py37hdbf6ddf_7']
tsnecuda -> numpy[version='>=1.14.1'] -> numpy-base[version='1.14.3|1.14.4|1.14.4|1.14.4|1.14.4|1.14.4|1.14.4|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.5|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.14.6|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.0|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.1|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.2|1.15.3|1.15.3|1.15.3|1.15.4|1.15.4|1.15.4|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.0|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.1|1.16.2|1.16.2|1.16.2|1.16.3|1.16.3|1.16.3|1.16.4|1.16.4|1.16.4',build='py36h2f8d375_0|py37h2f8d375_0|py36h2f8d375_0|py36h2f8d375_0|py27h2f8d375_0|py37h2f8d375_1|py37h2f8d375_0|py36h2f8d375_1|py37h2f8d375_1|py36h2f8d375_1|py36h2f8d375_0|py37h2f8d375_0|py36h2f8d375_0|py36h2f8d375_0|py36h2f8d375_1|py36h2f8d375_0|py27h2f8d375_0|py37h74e8950_0|py36h81de0dd_0|py35h81de0dd_0|py36h7cdd4dd_0|py36h3dfced4_0|py35h7cdd4dd_0|py35h3dfced4_0|py37h81de0dd_4|py37h2f8d375_4|py36h2f8d375_4|py27h81de0dd_4|py27h2f8d375_4|py37hdbf6ddf_4|py37hdbf6ddf_1|py37h2b20989_3|py37h2b20989_1|py36hdbf6ddf_4|py36hdbf6ddf_3|py36hdbf6ddf_2|py36h2b20989_3|py36h2b20989_2|py36h2b20989_0|py35hdbf6ddf_4|py35h2b20989_4|py27hdbf6ddf_4|py27h2b20989_4|py27h2b20989_3|py27h2b20989_0|py36h2b20989_0|py35hdbf6ddf_0|py27hdbf6ddf_0|py27h2b20989_0|py27h0ea5e3f_1|py27h9be14a7_1|py35h0ea5e3f_1|py35h9be14a7_1|py36h0ea5e3f_1|py36h9be14a7_1|py35h2b20989_0|py36hdbf6ddf_0|py27h2b20989_1|py27h2b20989_2|py27hdbf6ddf_0|py27hdbf6ddf_1|py27hdbf6ddf_2|py27hdbf6ddf_3|py35hdbf6ddf_0|py36h2b20989_1|py36h2b20989_4|py36hdbf6ddf_0|py36hdbf6ddf_1|py37h2b20989_2|py37h2b20989_4|py37hdbf6ddf_2|py37hdbf6ddf_3|py35h2f8d375_4|py35h81de0dd_4|py36h81de0dd_4|py27h3dfced4_0|py27h7cdd4dd_0|py37h3dfced4_0|py37h7cdd4dd_0|py27h2f8d375_0|py27h74e8950_0|py27h81de0dd_0|py35h2f8d375_0|py35h74e8950_0|py36h2f8d375_0|py36h74e8950_0|py37h2f8d375_0|py37h81de0dd_0|py27h2f8d375_1|py35h2f8d375_0|py37h2f8d375_0|py37h2f8d375_1|py27h2f8d375_0|py37h2f8d375_0|py27h2f8d375_0|py27h2f8d375_0|py27h2f8d375_1|py37h2f8d375_0|py27h2f8d375_0|py27h2f8d375_1|py36h2f8d375_0|py37h2f8d375_0|py27h2f8d375_0|py27h2f8d375_0|py37h2f8d375_0']
mkl_random -> numpy-base[version='>=1.0.2,<2.0a0']
numpy-base
Package libopenblas conflicts for:
tsnecuda -> libopenblas
Any hints?
Cheers
DavidMChan commented
The reason for this is that tsnecuda uses a frozen version of FAISS as part of the compilation version for it's package. Because we have to slightly adapt the libraries, we don't use conda's FAISS GPU as a dependency. Regarding your first issue, I can't seem to reproduce this on our machines.
Can you determine if libfaiss.so is somewhere in the directory: .../anaconda/lib/pythonX.X/site-packages/tsnecuda/? It seems like while tsnecuda is installed, it's not able to see the generated library.
samuelbroscheit commented
Ah ok. Apparently I jumped too fast on installing faiss-gpu. I cannot reproduce the libfaiss.so error as well. Thanks.