containerd/runwasi

Adding containerd config options for the shim

jsturtevant opened this issue · 9 comments

jsturtevant commented

Consider adding containerd config options for the shim to enable feature code paths. Adding containerd config options for the shim would allow for on node configuration and debugging. It could also provide a path for turning on and off newly release / unstable features w/o recompilation.

Originally from spinkube/containerd-shim-spin#79 (comment)

Mossaka commented

I have a couple questions for this ask:

  1. What kind of options does containerd support to parse? I saw this section of the doc mentions that you can add options to shim runtime (e.g. through [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.<runtime>.options]) but isn't clear on what options are allowed and how will these options being feed to the shim.
  2. What options do we (runwasi) want to support? I would imagine that there is a diff among the set of options that are allowed for different shims (e.g. wasmtime shim vs. spin shim). Or do these options largely open to interpretation for each shim implementaion?
    To elaborate a bit more. I think options like "enable logs" or "enable tracing" are kind of common across all shims, so perhaps that's something runwasi should provide? On the other hand, options like "disable wasmtime precompilation" is a wasmtime shim specific option, which should not be of a concern of runwasi.
  3. Do users have to restart containerd everytime to enable / disable an option for a shim?
    • Follow up on the question above, I think it's best to let runtime-class-manager operator (e.g. the design doc by @devigned ) to mutate the configurations for each shim.
kate-goldenring commented

As @jprendes mentioned, it looks like the options set in the containerd config should be passed in the CreateTaskRequest message under the options field. This is received during create request (issued by containerd) in runwasi. To test out configuring options with containerd, I added an option to the config.toml for the Spin shim and restarted the containerd service:

 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.spin]
    runtime_type = "io.containerd.spin.v2"

  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.spin.options]
    Foo = "bar"

I then built the shim locally with a print statement here in runwasi:

    fn task_create(&self, req: CreateTaskRequest) -> Result<CreateTaskResponse> {
        log::info!("task_create called with options: {:?}", req.options);

I expected the option to be passed by containerd and logged but there are still not options set:

May 17 16:39:35 kagold-ThinkPad-X1-Carbon-6th containerd[738]: time="2024-05-17T23:39:35.20109351Z" level=info msg="task_create;"
May 17 16:39:35 kagold-ThinkPad-X1-Carbon-6th containerd[738]: time="2024-05-17T23:39:35.201106634Z" level=info msg="task_create called with options: MessageField(None)"

Does anyone have any recommendations for how to approach this? runc passes the following options:

          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = true
Mossaka commented

I expected the option to be passed by containerd and logged but there are still not options set:

How did you run the shim? Did you use ctr or kubectl? It might be that containerd only applies configuration to the shim only via CRI (see https://github.com/containerd/containerd/blob/main/docs/man/containerd-config.toml.5.md#multiple-runtimes)

kate-goldenring commented

Thats a good call out. I'll try again from a Kubernetes context

jsturtevant commented
  • What options do we (runwasi) want to support? I would imagine that there is a diff among the set of options that are allowed for different shims (e.g. wasmtime shim vs. spin shim). Or do these options largely open to interpretation for each shim implementaion?
    To elaborate a bit more. I think options like "enable logs" or "enable tracing" are kind of common across all shims, so perhaps that's something runwasi should provide? On the other hand, options like "disable wasmtime precompilation" is a wasmtime shim specific option, which should not be of a concern of runwasi.

One set of options we should consider is how to turn on various runtime options as mentioned in spinkube/containerd-shim-spin#115 (comment)

kate-goldenring commented

When using CRI, I was able to see that runwasi receives the options 🎉 So, we'd need to parse the received object and decide where to pass it forward.

  • 🎉3