Unable to install satnet through pip due to compilation error

Question

Unable to install satnet through pip due to compilation error

gbrlfaria opened this issue 2 years ago · 5 comments

Hello,

I was trying to install satnet using pip, but I ran into some difficulty. It appears that the compiler is unable to locate the necessary header files, even though the ./src directory is included in include_dirs, and is even visible in the -I compilation flag.

I am currently using the provided Docker environment within WSL, but I have encountered the same problem when using Ubuntu as well.

Luckily, I was finally able to set up the package by cloning the repository and installing the package locally instead.

In any case, do you have any insight into what might be causing this issue? I have attached the output I'm getting.

Thank you.

work@docker-desktop:/data$ pip install satnet
Defaulting to user installation because normal site-packages is not writeable
Collecting satnet
  Using cached satnet-0.1.4.tar.gz (10 kB)
Requirement already satisfied: torch>=1.3 in /opt/conda/lib/python3.9/site-packages (from satnet) (1.13.0)
Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.9/site-packages (from torch>=1.3->satnet) (4.4.0)
Building wheels for collected packages: satnet
  Building wheel for satnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-wvjy7sui
       cwd: /tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/
  Complete output (20 lines):
  running bdist_wheel
  /opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.9
  creating build/lib.linux-x86_64-3.9/satnet
  copying satnet/models.py -> build/lib.linux-x86_64-3.9/satnet
  copying satnet/__init__.py -> build/lib.linux-x86_64-3.9/satnet
  running build_ext
  building 'satnet._cpp' extension
  creating build/temp.linux-x86_64-3.9
  creating build/temp.linux-x86_64-3.9/src
  gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -I/opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -I./src -I/opt/conda/lib/python3.9/site-packages/torch/include -I/opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.9/site-packages/torch/include/TH -I/opt/conda/lib/python3.9/site-packages/torch/include/THC -I/opt/conda/include/python3.9 -c src/satnet.cpp -o build/temp.linux-x86_64-3.9/src/satnet.o -fopenmp -msse4.1 -Wall -g -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=_cpp -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  src/satnet.cpp:28:10: fatal error: satnet.h: No such file or directory
   #include "satnet.h"
            ^~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for satnet
  Running setup.py clean for satnet
Failed to build satnet
Installing collected packages: satnet
    Running setup.py install for satnet ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-98iunexk/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/work/.local/include/python3.9/satnet
         cwd: /tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/
    Complete output (22 lines):
    running install
    /opt/conda/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.9
    creating build/lib.linux-x86_64-3.9/satnet
    copying satnet/models.py -> build/lib.linux-x86_64-3.9/satnet
    copying satnet/__init__.py -> build/lib.linux-x86_64-3.9/satnet
    running build_ext
    /opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    building 'satnet._cpp' extension
    creating build/temp.linux-x86_64-3.9
    creating build/temp.linux-x86_64-3.9/src
    gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -I/opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -I./src -I/opt/conda/lib/python3.9/site-packages/torch/include -I/opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.9/site-packages/torch/include/TH -I/opt/conda/lib/python3.9/site-packages/torch/include/THC -I/opt/conda/include/python3.9 -c src/satnet.cpp -o build/temp.linux-x86_64-3.9/src/satnet.o -fopenmp -msse4.1 -Wall -g -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=_cpp -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    src/satnet.cpp:28:10: fatal error: satnet.h: No such file or directory
     #include "satnet.h"
              ^~~~~~~~~~
    compilation terminated.
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yw2gliog/satnet_0e8bceaba4ac4719931aac6e4b35b01d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-98iunexk/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/work/.local/include/python3.9/satnet Check the logs for full command output.

Answer 1 · 2023-01-19T14:06:10.000Z

Mostly it should be related to how PyTorch deal with the include_dirs in the CppExtension. What's the environment you're using? (Pytorch & Python version, the WSL docker tag).

Answer 2 · 2023-01-19T20:54:31.000Z

Thanks for the quick reply. Here is my environment (please tell me if I missed something):

WSL2:

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

Docker image:

satnet:latest (ubuntu:18.04, bionic, bionic-20221019)

Python version:

Python 3.9.12 (main, Apr  5 2022, 06:56:58) 
[GCC 7.5.0] :: Anaconda, Inc. on linux

PyTorch version:

pytorch             1.13.0      py3.9_cuda11.6_cudnn8.3.2_0     pytorch
pytorch-cuda        11.6        h867d48c_0                      pytorch
pytorch-mutex       1.0         cuda                            pytorch
torchtext           0.14.0      py39                            pytorch
torchvision         0.14.0      py39_cu116                      pytorch

Answer 3 · 2023-03-31T20:20:05.000Z

I'm having this error when running pip install or build or python install. Does anyone has any idea?

cuda_11.2.r11.2
Python 3.7.9
tensorflow '2.10.0'
torch '1.11.0+cu113'

Answer 4 · 2023-04-01T09:17:17.000Z

It's mostly the path/lib problem for the NVCC toolchain on Windows. To try the package, I recommend using it on Google Colab. If there is a necessity to run it on Windows, a good start for debugging is to clone it locally & do a verbose output when doing the make_ext.

Answer 5 · 2023-04-02T01:45:36.000Z

It's mostly the path/lib problem for the NVCC toolchain on Windows. To try the package, I recommend using it on Google Colab. If there is a necessity to run it on Windows, a good start for debugging is to clone it locally & do a verbose output when doing the make_ext.

Actually, extracting the full log error, I realized, it was a problem with open mp, on the compilation of satnet._cpp the command -fopenmp wasn't recognized by MSoft visual studio. I guess that it's for linux compilation.

But I tried on colab, and it worked perfectly! Thanks!