google/cld3

Cannot install gcld3 on OS X

erip opened this issue · 4 comments

erip commented

Hi all. I have protobuf 3.15.8 installed but when I try to install gcld3 through pip, protobuf headers aren't found when building from source:

$ pip install gcld3
Collecting gcld3
  Using cached gcld3-3.0.13.tar.gz (647 kB)
Building wheels for collected packages: gcld3
  Building wheel for gcld3 (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/erip/miniconda3/envs/langid/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/setup.py'"'"'; __file__='"'"'/private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-wheel-jab5wg99
       cwd: /private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/
  Complete output (25 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.7
  creating build/lib.macosx-10.9-x86_64-3.7/gcld3
  copying gcld3/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/gcld3
  running build_ext
  building 'gcld3.pybind_ext' extension
  creating build/temp.macosx-10.9-x86_64-3.7
  creating build/temp.macosx-10.9-x86_64-3.7/gcld3
  creating build/temp.macosx-10.9-x86_64-3.7/src
  creating build/temp.macosx-10.9-x86_64-3.7/src/cld_3
  creating build/temp.macosx-10.9-x86_64-3.7/src/cld_3/protos
  creating build/temp.macosx-10.9-x86_64-3.7/src/script_span
  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/erip/miniconda3/envs/langid/include -arch x86_64 -I/Users/erip/miniconda3/envs/langid/include -arch x86_64 -I/Users/erip/miniconda3/envs/langid/lib/python3.7/site-packages/pybind11/include -I/Users/erip/miniconda3/envs/langid/include/python3.7m -c gcld3/pybind_ext.cc -o build/temp.macosx-10.9-x86_64-3.7/gcld3/pybind_ext.o -std=c++11 -stdlib=libc++
  In file included from gcld3/pybind_ext.cc:5:
  In file included from gcld3/../src/nnet_language_identifier.h:22:
  In file included from gcld3/../src/embedding_feature_extractor.h:23:
  In file included from gcld3/../src/feature_extractor.h:45:
  gcld3/../src/cld_3/protos/feature_extractor.pb.h:9:10: fatal error: 'google/protobuf/stubs/common.h' file not found
  #include <google/protobuf/stubs/common.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  1 error generated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for gcld3
  Running setup.py clean for gcld3
Failed to build gcld3
Installing collected packages: gcld3
    Running setup.py install for gcld3 ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/erip/miniconda3/envs/langid/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/setup.py'"'"'; __file__='"'"'/private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-record-20khe1jh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/erip/miniconda3/envs/langid/include/python3.7m/gcld3
         cwd: /private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/
    Complete output (25 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.9-x86_64-3.7
    creating build/lib.macosx-10.9-x86_64-3.7/gcld3
    copying gcld3/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/gcld3
    running build_ext
    building 'gcld3.pybind_ext' extension
    creating build/temp.macosx-10.9-x86_64-3.7
    creating build/temp.macosx-10.9-x86_64-3.7/gcld3
    creating build/temp.macosx-10.9-x86_64-3.7/src
    creating build/temp.macosx-10.9-x86_64-3.7/src/cld_3
    creating build/temp.macosx-10.9-x86_64-3.7/src/cld_3/protos
    creating build/temp.macosx-10.9-x86_64-3.7/src/script_span
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/erip/miniconda3/envs/langid/include -arch x86_64 -I/Users/erip/miniconda3/envs/langid/include -arch x86_64 -I/Users/erip/miniconda3/envs/langid/lib/python3.7/site-packages/pybind11/include -I/Users/erip/miniconda3/envs/langid/include/python3.7m -c gcld3/pybind_ext.cc -o build/temp.macosx-10.9-x86_64-3.7/gcld3/pybind_ext.o -std=c++11 -stdlib=libc++
    In file included from gcld3/pybind_ext.cc:5:
    In file included from gcld3/../src/nnet_language_identifier.h:22:
    In file included from gcld3/../src/embedding_feature_extractor.h:23:
    In file included from gcld3/../src/feature_extractor.h:45:
    gcld3/../src/cld_3/protos/feature_extractor.pb.h:9:10: fatal error: 'google/protobuf/stubs/common.h' file not found
    #include <google/protobuf/stubs/common.h>
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1 error generated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/erip/miniconda3/envs/langid/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/setup.py'"'"'; __file__='"'"'/private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-install-gsc_bqq7/gcld3_2a1eb194f3294e64ad0609ee476c9a1a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/v8/ls8m50ks3zz0hzq21681r8m40000gn/T/pip-record-20khe1jh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/erip/miniconda3/envs/langid/include/python3.7m/gcld3 Check the logs for full command output.

Do you have any thoughts?

Kristine1975 commented

I just installed gcld3 successfully on macOS 10.14.6. First I installed protobuf through homebrew (brew install protobuf), then I installed gcld3 itself through pip (pip3 install gcld3).

It seems that you installed protobuf in a directory that's not part of gcc's default include directories (where gcc searches for header files such as above google/protobuf/stubs/common.h ). You can get gcc's default include directories by executing gcc -xc++ -E -v -.

You can add to gcc's default include directories by using the environment variables C_INCLUDE_PATH and CPLUS_INCLUDE_PATH. I haven't tried that myself, though.

efung commented

Just a followup related to the previous comment, for those on macOS 12.1 and/or using M1 Silicon:

On M1 (ARM) Macs, after installing protobuf via Homebrew, the headers will be located in /opt/homebrew/include. So you'll need to add this path to CPLUS_INCLUDE_PATH or CPATH, depending on which compiler you're using.

For example, if using clang from the Xcode Command Line Tools, you would install via:

$ CPATH=/opt/homebrew/include pip3 install gcld3
sschamp commented

For me, this is what worked on an M1 Mac:

brew install protobuf
CPLUS_INCLUDE_PATH=/opt/homebrew/include LIBRARY_PATH=/opt/homebrew/Cellar/protobuf/21.12/lib pip3 install gcld3

Optionally:

brew install gcc llvm