faiss installed from the channel conda-forge breaks tsnecuda
hkunzhe opened this issue · 3 comments
Since faiss-gpu doesn't provide compiled binaries with cuda10.1, 10.2, and 11.0, see this issue for details. Thus, I install faiss from the channel conda-forge by:
There are some already compiled builds for cuda 10.1, 10.2, and 11.0 in the conda-forge channel. To install a specific build, try conda search faiss= --info -c conda-forge and find the no-conflict one with your environment. Then you can install by conda install faiss==<build_string>, e.g., conda install faiss=1.6.3=py37hddabd6f_3_cuda -c conda-forge.
However, when I run:
import tsnecuda
tsnecuda.test()
It shows an OSError: anaconda3/envs/test/lib/python3.7/site-packages/tsnecuda/libtsnecuda.so: undefined symbol: _ZTIN5faiss11IndexShardsE.
Steps to reproduce:
- conda create -n test python=3.7 and conda activate test.
- conda install tsnecuda cuda101 -c cannylab.
- conda install faiss=1.6.3=py37hddabd6f_3_cuda -c conda-forge.
- run
tsnecuda.test().
It's a similar issue with #65. I've tested faiss-gpu with cudatoolkit 10.0 and tsnecuda with cuda100 working fine.
Yes - I think it remains outstanding to build our latest version of the code linking dynamically against the faiss-gpu libraries from conda-forge.
Our current build from source will build faiss-GPU in addition to tsnecuda, linking against the local version. If you have a second version of faiss resident in conda, I'm not sure exactly what happens - it likely tries to link against the conda-forge version, which is different from the static symbol compiled into the local version of the code.
The best solution seems to be to build tsnecuda from source, and alter the CMakeLists.txt to link against faiss-GPU as a dynamic library instead of linking statically against a locally compiled version. At the time we wrote this library, this wasn't an option, but it seems like it may be possible now.
Ok, I got it! Thank you very much:)