microsoft/spacy-ann-linker

[BUG] Error importing nmslib in Python 3.8

r-b-g-b opened this issue · 2 comments

Describe the bug

This is a bug in the nmslib version pinned in spacy-ann-linker requirements. In Python 3.8.7 on Linux, importing nmslib fails with the following error:

ImportError: Object of type 'Logger' is not an instance of 'module_'

To Reproduce

The simplest way to reproduce is to only install nmslib==2.0.5. You can use the Python docker image:

docker \
    run \
    -it \
    --entrypoint bash \
    python:3.8
pip install nmslib==2.0.5

Output:

Collecting nmslib==2.0.5
  Using cached nmslib-2.0.5-cp38-cp38-linux_x86_64.whl
Collecting pybind11>=2.2.3
  Using cached pybind11-2.6.2-py2.py3-none-any.whl (191 kB)
Collecting numpy>=1.10.0
  Using cached numpy-1.20.1-cp38-cp38-manylinux2010_x86_64.whl (15.4 MB)
Collecting psutil
  Using cached psutil-5.8.0-cp38-cp38-manylinux2010_x86_64.whl (296 kB)
Installing collected packages: pybind11, psutil, numpy, nmslib
Successfully installed nmslib-2.0.5 numpy-1.20.1 psutil-5.8.0 pybind11-2.6.2
python -c 'import nmslib'

Output:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: Object of type 'Logger' is not an instance of 'module_'

Environment

  • OS: Debian GNU/Linux 10 (buster)
  • Typer Version: 0.3.2
  • Python version: 3.8.7

Additional context

  • There is a discussion of this error in the nmslib issues. There I saw one fix that worked, which is to downgrade pybind11 from 2.6.2 to 2.6.1 and install both from source. So pip install --no-binary :all: nmslib==2.0.5 pybind11==2.6.1 results in a working nmslib. However, it seems like the primary developer's solution was to release a new nmslib version 2.1.1 that fixes the problem. I have confirmed that pip install nmslib==2.1.1 fixes the issue here. I can also confirm that nmslib==2.1.1 passes all of our package tests, so maybe a solution is to upgrade the pinned version of nmslib to 2.1.1 in spacy-ann-linker (or upgrade/relax to nmslib>2.1).
  • Everything works fine if I use a conda environment. conda create -n py38 python=3.8, conda activate py38, pip install nmslib==2.0.5, nmslib imports fine. I haven't had a chance to dig deeper, except to confirm that it is using the same version of pybind11==2.6.2 that breaks in a non-conda setup that the Docker Python uses.

This should be fixed for real by nmslib/nmslib#488.

Thank you, it does seem that pinning nmslib >= 2.1.1, <2.2 as done in d6a785c fixes that error. I can now install spacy_ann_linker using:

pip install git+https://github.com/microsoft/spacy-ann-linker

Could I request new PyPI release that incorporates the latest code? That way I could install directly from PyPI. Thanks!