Plugin requirements
cmluciano opened this issue · 11 comments
Does this plugin function due to a limitation of runC or containerd settings? Is it possible to configure the prestart hooks at the containerd daemon settings layer instead or replacing runC with this plugin?
We use this patched version of runc to add our custom hook because we have no way of doing this with the current Docker API, e.g. see this request: moby/moby#28837
If you have a way to add our hook to the OCI runtime spec, then you don't need our runc fork, provided your version of runc is also recent enough to include this patch: opencontainers/runc#1239
For instance, Project Atomic's fork of Moby includes a patch to add custom hooks:
https://github.com/projectatomic/docker/tree/docker-17.03.1-ce#add-dockerhooks-exec-custom-hooks-for-prestartpoststop-containerspatch
@cmluciano did it answer your question?
Yes, I'm wondering if this is necessary within the k8s space though. This is a hard dependency on the k8s device plugin and I wonder why this could not be done with the device plugin.
why this could not be done with the device plugin
What do you mean?
In k8s, one can populate these values within the pod using an admission controller or at another level with the device-plugin. Is there a reason these need to be patched in at the runC pre-hook level?
The current device plugin API doesn't allow you to inject hooks.
@Fix42 would a proxy in-front of runc that modified the config file to add the hooks work and avoid the need for a custom runc?
@flx42 does the nvidia prestart hook require the container's namespaces/pid/... to already be setup prior to the hook doings its job?
would a proxy in-front of runc that modified the config file to add the hooks work and avoid the need for a custom runc?
If your version of runc is recent enough to include opencontainers/runc#1239, then yes it would work, otherwise you will need to backport this patch like we do.
In the case of Docker, it will still require to register a new runtime to the daemon.
This would still require reading, parsing and writing the new configuration file on the filesystem.
If you have full control over the spec, you should definitely skip nvidia-container-runtime and inject nvidia-container-runtime-hook directly.
does the nvidia prestart hook require the container's namespaces/pid/... to already be setup prior to the hook doings its job?
Note that the hook only works for OCI-compatible runtimes, therefore I'm not sure I understand why you are asking this question. Maybe you are asking about nvidia-container-cli (based on libnvidia-container) in general. nvidia-container-cli can configure a target PID (with --pid=), or the current PID. In both cases the mount namespace needs to be setup, but obviously you need to call nvidia-container-cli before the pivot_root.
cc @3XX0