docker-compose support
DarioTurchi opened this issue ยท 14 comments
1. Issue or feature description
not able to run containers with GPU acceleration from docker compose
With nvidia-docker2 I was able to run GPU accelerated containers by defining the nvidia
runtime within the file /etc/docker/daemon.json
:
{
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
and then use the runtime
key within the docker compose file (or set the nvidia
runtime as the default one)
With the new version of Nvidia-docker, nvidia container toolkit
this doesn't work anymore, since a dedicated runtime should not been run anymore, instead the flag --gpus
needs to be provided (or at least this is my understanding).
However, the wiki still suggests the 'runtime' solution, is that right ? Furthermore, the current installation doesn't provide the `/usr/bin/nvidia-container-runtime' file anymore
How does the current version of nvidia-docker should be integrated with docker-compose ?
Sorry if I didn't properly report this as an issue: it is indeed more a lack of documentation and I didn't know where to report it
docker-compose version 1.24.1
Docker version 19.03.2, build 6a30dfc
Exactly this, how can one use docker-compose
with the new nvidia-docker versions?
I'm encountering the same problem. Putting "runtime:nvidia" in docker-compose does not work, even if I follow the instructions from wiki.
Same problem - runtime: nvidia
in docker-compose.yml
doesn't work, making changes to /etc/docker/daemon.json
the way it's documented used to work, but doesn't now. docker run --gpus all
works, but replacing runtime: nvidia
with gpus: all
in docker-compose.yml
is invalid.
Hello!
Docker-compose support still works with the nvidia-docker2
package. You can set the default runtime to nvidia as indicated in the documentation.
What I think you are asking is does docker-compose support the `--gpus option? The answer is not yet!
@RenaudWasTaken Per the documentation it says to set it this way
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
},
}
In the newest version of nvidia-docker, there is no /usr/bin/nvidia-container-runtime
You need to install the old nvidia-docker2
packages :)
So in order to work with docker-compose with the latest nvidia-docker, I need to also install the older nvidia-docker2?
Nope, what I'm saying above is that the support in docker-compose for --gpus
is not here yet.
But you can still use the support for --runtime
by installing the nvidia-docker2
packages.
The great thing with the old packages is that you also get --gpus
for free :)
What do you mean "by installing nvidia-docker
packages"? If I followed all the instructions on the main GitHub page for a fresh installation, haven't I installed the nvidia-docker2
packages?
On the readme:
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
$ sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
$ sudo systemctl restart docker
Replace the line:
$ sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
By:
$ sudo apt-get update && sudo apt-get install -y nvidia-docker2
More detail discussion and progress are in Compose's issue.
docker/compose#6691
I tried nvidia-docker2
as suggested by @RenaudWasTaken with the following /etc/docker/daemon.json
file and then docker-compose up
works well.
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
this is my daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
docker run --runtime=nvidia works fine
docker-compose up won't work
Unsupported config option for xxx: 'runtime'
@leon0707 You have to use at least version 2.3 in your docker-compose.yml. You might have to update docker-compose to be able to do that: https://docs.docker.com/compose/install/