Error in loading new language identification models

[spate141]

fastText just released new language identification models with subword information in supervised learning models. here > adding subwords for supervised models

I tried loading these new models with pyfasttext library and it's giving me this error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-4-9054ed9f267e> in <module>()
----> model = FastText('/opt/models/lid.176.bin', label='__label__')

src/pyfasttext.pyx in pyfasttext.FastText.__cinit__ (src/pyfasttext.cpp:2050)()

src/pyfasttext.pyx in pyfasttext.FastText.load_model (src/pyfasttext.cpp:7386)()

RuntimeError: vector::_M_default_append

I believe this is because new changes in original fastText library?

[vrasneur]

I have released a new version (0.4.0) that supports the new models.

Can you update pyfasttext by using this command: pip install -U pyfasttext ?

[spate141]

@vrasneur perfect! everything is working, I am able to load new models.

Although new updated package installed successfully, but it produced some error, just if you want to take a look:

Collecting pyfasttext
  Downloading pyfasttext-0.4.0.tar.gz (213kB)
    100% |████████████████████████████████| 215kB 4.5MB/s 
Collecting cysignals (from pyfasttext)
  Downloading cysignals-1.6.5.tar.bz2 (69kB)
    100% |████████████████████████████████| 71kB 10.0MB/s 
Requirement already up-to-date: future in /root/anaconda2/lib/python2.7/site-packages/future-0.16.0-py2.7.egg (from pyfasttext)
Requirement already up-to-date: numpy in /root/anaconda2/lib/python2.7/site-packages (from pyfasttext)
Building wheels for collected packages: pyfasttext, cysignals
  Running setup.py bdist_wheel for pyfasttext ... error
  Complete output from command /root/anaconda2/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-B_nFd5/pyfasttext/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpF7Sdk9pip-wheel- --python-tag cp27:
  src/pyfasttext.pyx: cannot find cimported module 'cysignals.signals'
  running bdist_wheel
  running build
  running build_ext
  building 'pyfasttext' extension
  creating build
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/src
  creating build/temp.linux-x86_64-2.7/src/fastText
  creating build/temp.linux-x86_64-2.7/src/fastText/src
  gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -iquote . -include src/custom_exit.h -fPIC -I. -Isrc/variant/include -I/root/anaconda2/lib/python2.7/site-packages/numpy/core/include -I/root/anaconda2/include/python2.7 -c src/pyfasttext.cpp -o build/temp.linux-x86_64-2.7/src/pyfasttext.o -std=c++0x -Wno-sign-compare
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  In file included from /root/anaconda2/include/python2.7/Python.h:8:0,
                   from src/pyfasttext.cpp:57:
  /root/anaconda2/include/python2.7/pyconfig.h:1190:0: warning: "_POSIX_C_SOURCE" redefined
   #define _POSIX_C_SOURCE 200112L
   ^
  In file included from /usr/include/x86_64-linux-gnu/c++/5/bits/os_defines.h:39:0,
                   from /usr/include/x86_64-linux-gnu/c++/5/bits/c++config.h:482,
                   from /usr/include/c++/5/cstdlib:41,
                   from ./src/custom_exit.h:5,
                   from <command-line>:0:
  /usr/include/features.h:228:0: note: this is the location of the previous definition
   # define _POSIX_C_SOURCE 200809L
   ^
  In file included from /root/anaconda2/include/python2.7/Python.h:8:0,
                   from src/pyfasttext.cpp:57:
  /root/anaconda2/include/python2.7/pyconfig.h:1212:0: warning: "_XOPEN_SOURCE" redefined
   #define _XOPEN_SOURCE 600
   ^
  In file included from /usr/include/x86_64-linux-gnu/c++/5/bits/os_defines.h:39:0,
                   from /usr/include/x86_64-linux-gnu/c++/5/bits/c++config.h:482,
                   from /usr/include/c++/5/cstdlib:41,
                   from ./src/custom_exit.h:5,
                   from <command-line>:0:
  /usr/include/features.h:169:0: note: this is the location of the previous definition
   # define _XOPEN_SOURCE 700
   ^
  src/pyfasttext.cpp:556:28: fatal error: struct_signals.h: No such file or directory
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for pyfasttext
  Running setup.py clean for pyfasttext
  Running setup.py bdist_wheel for cysignals ... done
  Stored in directory: /root/.cache/pip/wheels/3e/b8/c4/567c8c51996e21a25dc68a376dcb812a16070f8da5df0bcb9a
Successfully built cysignals
Failed to build pyfasttext
Installing collected packages: cysignals, pyfasttext
  Found existing installation: pyfasttext 0.3.0
    Uninstalling pyfasttext-0.3.0:
      Successfully uninstalled pyfasttext-0.3.0
  Running setup.py install for pyfasttext ... done
Successfully installed cysignals-1.6.5 pyfasttext-0.4.0