Samagra-Development/ai-tools

Make docker container based deployments use GPU, if GPUs are available on host system.

Closed this issue · 1 comments

The current deployment does not configure the compose to use Host GPUs if available, add that support.

Referenece: https://docs.docker.com/compose/gpu-support/#:~:text=GPUs%20are%20referenced%20in%20a,capabilities%20.

Related to #218

Steps:

  1. Ensure NVIDIA GPU Drivers are installed (Machine reboot might be required as well)
curl -s -L https://nvidia.github.io/nvidia-container-runtime/gpgkey | \
  sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-container-runtime/$distribution/nvidia-container-runtime.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
sudo apt-get update
sudo apt-get install nvidia-container-runtime
  1. Update /etc/docker/daemon.json to use nvidia as the default runtime
{
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}
  1. Restart Docker

sudo service docker restart

  1. Set the below environment variable to allow container to use GPU
NVIDIA_VISIBLE_DEVICES=all