docker-compose with nvidia cuda gpu support can't see 'libcuda.so'

[smolendawid]

After I run
sudo docker-compose --env-file ~/env.dev -f docker-compose-gpu.yaml up -d

My output is:

Removing vr
Recreating 6f15456889f0_vr ...
Recreating 6f15456889f0_vr ... error

ERROR: for 6f15456889f0_vr Cannot start service vr: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: mount error: stat failed: /var/lib/docker/overlay2/3e69c4833ad6cf72748f7f259e8a408d4f567593d68432f69c8a5ae25a8e078e/merged/usr/local/cuda-10.2/compat/libcuda.so.440.118.02: no such file or directory: unknown

ERROR: for vr Cannot start service vr: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: mount error: stat failed: /var/lib/docker/overlay2/3e69c4833ad6cf72748f7f259e8a408d4f567593d68432f69c8a5ae25a8e078e/merged/usr/local/cuda-10.2/compat/libcuda.so.440.118.02: no such file or directory: unknown
ERROR: Encountered errors while bringing up the project.

Output of nvidia-smi is:

Every 2.0s: nvidia-smi                                                                                                       ip-172-31-75-228: Tue Mar 23 07:55:57 2021

Tue Mar 23 07:55:57 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |

Unfortunately, I have no idea what is the problem and where to look for the solution. Any hints appreciated!

[elezar]

If this is still a problem, please open an issue against the https://github.com/NVIDIA/nvidia-container-toolkit repository.