Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.8

[leestott]

Error:

Failed loading model mistral-7b-v02-int4-gpu: /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1426 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory

Workaround to resolve issue

Check cuDNN Installation:

First run updates on all packages

sudo apt update
sudo apt upgrade
Ensure that you have installed cuDNN correctly. You can download the cuDNN library from the NVIDIA website and follow the installation guide.

Install cuda drivers and onnxruntime

pip install onnxruntime
pip install onnxruntime-gpu
Make sure the library is in the expected location (usually /usr/local/cuda/lib64).

Check LD_LIBRARY_PATH:Set the LD_LIBRARY_PATH environment variable to include the directory containing libcudnn.so.8. For example:

export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
Verify CUDA Toolkit Version: Confirm that your installed CUDA Toolkit version matches the version expected by TensorFlow. You might need to adjust the CUDA version in your TensorFlow code or install a compatible version of cuDNN.

If you then get a error saying a specific version is missing i.e. libcudnn8 I recommend you manually install

Find if the library is installed

find / -type f -name "libcudnn.so.8" 2>/dev/null

You can then reinstall the specific version

sudo apt-get install libcudnn8

[leestott]

Resolved using these commands

• pip install onnxruntime
• pip install onnxruntime-gpu
• cd /usr/local/cuda/lib64
• ls
• sudo apt install nvidia-cudnn
• sudo apt update
• apt list --upgradable
• sudo apt upgrade
• sudo apt update
• sudo apt update —fix-missing
• sudo apt-get install libcudnn8
• sudo apt update