CUDA driver version is insufficient for CUDA runtime version
tastyminerals opened this issue · 2 comments
Reinstalling torch and attempting to run the model results in the following error:
THCudaCheck FAIL file=/home/pavel/torch/extra/cutorch/lib/THC/THCGeneral.c line=70 error=35 : CUDA driver version is insufficient for CUDA runtime version
/home/pavel/torch/install/bin/luajit: /home/pavel/torch/install/share/lua/5.1/trepl/init.lua:389: /home/pavel/torch/install/share/lua/5.1/trepl/init.lua:389: /home/pavel/torch/install/share/lua/5.1/trepl/init.lua:389: loop or previous error loading module 'cunn'
stack traceback:
[C]: in function 'error'
/home/pavel/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
main.lua:11: in main chunk
[C]: in function 'dofile'
...avel/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405c90
ldconfig -p | grep libcuda
libcudart.so.9.1 (libc6,x86-64) => /opt/cuda/lib64/libcudart.so.9.1
libcudart.so (libc6,x86-64) => /opt/cuda/lib64/libcudart.so
libcuda.so.1 (libc6,x86-64) => /usr/lib/libcuda.so.1
libcuda.so.1 (libc6) => /usr/lib32/libcuda.so.1
libcuda.so (libc6,x86-64) => /usr/lib/libcuda.so
libcuda.so (libc6) => /usr/lib32/libcuda.so
sudo ls -lR /var/lib/nvidia-docker | grep libcuda
lrwxrwxrwx 1 root root 17 25. Sep 11:43 libcuda.so -> libcuda.so.375.82
lrwxrwxrwx 1 root root 17 25. Sep 11:43 libcuda.so.1 -> libcuda.so.375.82
-rwxr-xr-x 1 root root 7792656 26. Jul 12:08 libcuda.so.375.82
lrwxrwxrwx 1 root root 17 25. Sep 11:43 libcuda.so -> libcuda.so.375.82
lrwxrwxrwx 1 root root 17 25. Sep 11:43 libcuda.so.1 -> libcuda.so.375.82
-rwxr-xr-x 1 root root 8241128 26. Jul 12:06 libcuda.so.375.82
lrwxrwxrwx 1 root root 17 24. Okt 10:34 libcuda.so -> libcuda.so.384.90
lrwxrwxrwx 1 root root 17 24. Okt 10:34 libcuda.so.1 -> libcuda.so.384.90
-rwxr-xr-x 1 root root 12265428 21. Sep 22:43 libcuda.so.384.90
lrwxrwxrwx 1 root root 17 24. Okt 10:34 libcuda.so -> libcuda.so.384.90
lrwxrwxrwx 1 root root 17 24. Okt 10:34 libcuda.so.1 -> libcuda.so.384.90
-rwxr-xr-x 1 root root 13038712 21. Sep 22:21 libcuda.so.384.90
This is not a Torch issue. This is NVIDIA sending its regards to all linux users: https://devtalk.nvidia.com/default/topic/1028320/cuda-driver-version-is-insufficient-for-cuda-runtime-version/?offset=6
Archlinux/Manjaro users roll back to previous version:
sudo pacman -U /var/cache/pacman/pkg/cuda-9.0.176-4-x86_64.pkg.tar.xz
Do not update to CUDA 9.1 just yet!
THCudaCheck FAIL file=/pytorch/torch/csrc/cuda/Module.cpp line=34 error=35 : CUDA driver version is insufficient for CUDA runtime version
Traceback (most recent call last):
File "run.py", line 54, in
torch.cuda.set_device(opt.gpu)
File "/home/kamran/paraphrase_sentence_regeneration/wean_github_code/wean_pyenv36/pyenv36/lib/python3.6/site-packages/torch/cuda/init.py", line 264, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: cuda runtime error (35) : CUDA driver version is insufficient for CUDA runtime version at /pytorch/torch/csrc/cuda/Module.cpp:34
[INFO/MainProcess] process shutting down