THC/THC.h: No such file or directory
cchan-lm opened this issue · 7 comments
cchan-lm commented
It appears since PyTorch 1.11, TH/THC include files have been removed. warp_rnnt wasn't the only one affected. Other repos relying on these include files were as well (see open-mmlab/mmdetection3d#1332).
Python version: 3.9.11
OS: Ubuntu 18.04.6 LTS
pip install torch==1.11.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
pip install warp_rnnt
Output:
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting warp_rnnt
Downloading warp_rnnt-0.5.0.tar.gz (10 kB)
Preparing metadata (setup.py) ... done
Collecting pybind11
Downloading pybind11-2.10.0-py3-none-any.whl (213 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 213.3/213.3 kB 32.3 MB/s eta 0:00:00
Collecting numpy
Downloading numpy-1.23.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.1/17.1 MB 10.9 MB/s eta 0:00:00
Requirement already satisfied: torch>=1.0.0 in ./.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/site-packages (from warp _rnnt) (1.12.1+cu113)
Requirement already satisfied: typing-extensions in ./.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/site-packages (from torch>=1.0.0->warp_rnnt) (4.3.0)
Using legacy 'setup.py install' for warp_rnnt, since package 'wheel' is not installed.
Installing collected packages: pybind11, numpy, warp_rnnt
Running setup.py install for warp_rnnt ... error
error: subprocess-exited-with-error
× Running setup.py install for warp_rnnt did not run successfully.
│ exit code: 1
╰─> [23 lines of output]
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/warp_rnnt
copying warp_rnnt/test.py -> build/lib.linux-x86_64-3.9/warp_rnnt
copying warp_rnnt/__init__.py -> build/lib.linux-x86_64-3.9/warp_rnnt
running build_ext
/home/linuxbrew/.linuxbrew/Cellar/pyenv/2.3.4/libexec/pyenv-exec: /home/craig/bin/ninja: /usr/local/bin/python: ba d interpreter: No such file or directory
/home/linuxbrew/.linuxbrew/Cellar/pyenv/2.3.4/libexec/pyenv-exec: line 48: /home/craig/bin/ninja: Success
/home/craig/.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/site-packages/torch/utils/cpp_extension.py:411: UserWar ning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
/home/craig/.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/site-packages/torch/utils/cpp_extension.py:813: UserWar ning: The detected CUDA version (11.1) has a minor version mismatch with the version that was used to compile PyTorch (1 1.3). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'warp_rnnt._C' extension
creating build/temp.linux-x86_64-3.9
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/craig/.pyenv/versions/3 .9.11/envs/rnnt/lib/python3.9/site-packages/torch/include -I/home/craig/.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/s ite-packages/torch/include/torch/csrc/api/include -I/home/craig/.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/site-pack ages/torch/include/TH -I/home/craig/.pyenv/versions/3.9.11/envs/rnnt/lib/python3.9/site-packages/torch/include/THC -I/us r/local/cuda/include -I/home/craig/.pyenv/versions/3.9.11/envs/rnnt/include -I/home/craig/.pyenv/versions/3.9.11/include /python3.9 -c binding.cpp -o build/temp.linux-x86_64-3.9/binding.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_T YPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CX X11_ABI=0 -std=c++14
binding.cpp:4:10: fatal error: THC/THC.h: No such file or directory
#include <THC/THC.h>
^~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> warp_rnnt
note: This is an issue with the package mentioned above, not pip.
The above error also occurs for PyTorch 1.12, which we're required to use for our software. Will warp_rnnt be updated, or do you recommend another way forward?
pooya-mohammadi commented
The same error here.
Do you have any plans to update the repository?
1ytic commented
Hey, thank you for your feedbacks. Could you try a new version pip install warp-rnnt==0.6.0? Should be solved now.
pooya-mohammadi commented
@1ytic
Thanks for the update. I get the following error:
139 | #error -- unsupported GNU version! gcc versions later than 10 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
1ytic commented
Please share your environment details: OS, Python version, PyTorch version, CUDA/cuDNN version, GCC version. Could you try another version of gcc?
pooya-mohammadi commented
OS: Ubuntu 22.04
python version: 3.9.12
cuda version: NVIDIA-SMI 510.85.02 Driver Version: 510.85.02 CUDA Version: 11.6
cudatoolkit: 11.3.1 -> installed by conda
cudnn: cudnn-linux-x86_64-8.3.3.40_cuda11.5
cudnn: py3.9_cuda11.3_cudnn8.2.0_0 installed by conda along with pytorch
gcc: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
pytorch: 1.11.0 -> installed by conda
Unfortunately, I cannot install another version of gcc
cchan-lm commented
Installation works on my end! Thank you for fixing this :) Leaving as open pending resolution for @pooya-mohammadi, but feel free to close afterwards!
pooya-mohammadi commented
@cchan-lm
would you mind reporting your environments details?