toolbox: use rkt fly instead of systemd-nspawn
coreosbot opened this issue · 8 comments
Issue by @philips
Today toolbox uses systemd-nspawn to actual execute the machine. The downside of systemd-nspawn is that there is no propagation. So, you can't say mount something inside of toolbox or unmount something that is seen in the root mount namespace.
Lets do that. Here is a prototype: https://gist.github.com/philips/acc5d812041f7a5baa2d582626cddb2a
xref rkt mount propagation rkt/rkt#1149
Comment by @dm0-
There are a few toolbox settings that seem to expect bare systemd-nspawn command-line arguments. If we switch to rkt, users' existing configuration files using these settings won't work:
TOOLBOX_BINDTOOLBOX_ENV
For TOOLBOX_ENV, I'd suggest ignoring it with rkt and adding a TOOLBOX_ENV_FILE option. Since rkt has --set-env-file, it can be used directly, but if the backend program changes again, an env file is trivial to parse and convert to whatever format is necessary.
I'd prefer to ignore TOOLBOX_BIND as well, and maybe add a generic TOOLBOX_RKT_ARGS setting for the same functionality.
There is also the TOOLBOX_DIRECTORY setting, which I believe can just be ignored since a rootfs no longer needs to be extracted for systemd-nspawn.
Any objections to these config changes?
Comment by @crawford
I think the config changes are fine. Toolbox has always been meant to be used interactively, so these flag changes should be manageable by the operator.
Comment by @crawford
I don't like the idea of a new tool. Unless it provides some ground-breaking functionality, I don't see a reason to further bloat the OS. Toolbox itself is going to stop working as we upgrade systemd (they removed --share-system). We should rip the bandaid off now.
Comment by @dm0-
@crawford They've actually just removed its documentation, but it's still there. https://github.com/systemd/systemd/blob/master/src/nspawn/nspawn.c#L443 Still, they're clearly moving towards removing it soon, so I agree we should deal with that. (I vaguely recall discussion of the CoreOS toolbox being its only user and the only reason for keeping the option.)
Comment by @lucab
@crawford @dm0- the outcome of the discussion with systemd upstream is that --share-system will disappear at some point, but the same functionality can anyway be toggled via granular env vars, see systemd/systemd#4023. rkt also needs some of those features for pods sharing some host namespaces.
Closing, since this is unlikely to happen at this point in CL's life cycle.