Torcx doesn't seem to work with systemd-nspawn containers

[dongsupark]

(This issue popped up during offline discussions with @lucab, so I'll write up here not to forget about it. At the moment for me this issue is not the highest priority.)

Torcx doesn't seem to work well with nspawn containers. Normally torcx wants /etc/torcx/next-profile to be specified for loading a user-defined profile. When the system then reboots, the next profile gets detected, tarball gets unpacked. But in nspawn containers, no matter how many times a container reboots, the next-profile never gets detected. Nothing but the predefined system-wide profile docker works for now. OTOH the next-profile mechanism works fine in a pure virtual machine.

It's also not trivial for me to define system-wide profile, because the docker profile is defined in coreos-overlay, which 3rd-party users won't be able to touch.

See also kinvolk/kube-spawn#105 (comment)

[lucab]

Thanks for the report. Can you please check the output of journalctl --boot 0 --identifier /usr/lib64/systemd/system-generators/torcx-generator? Or if missing any other mentions of torcx in the system journal.

[lucab]

I'm closing this, as I just double-checked on current ContainerLinux alpha and defining+applying a custom profile (e.g. to rollback docker) works as expected and override the default profile (with docker 18.02):

$ cat /etc/torcx/next-profile 
systemd-nspawn

$ cat /etc/torcx/profiles/systemd-nspawn.json 
{
  "kind": "profile-manifest-v0",
  "value": {
    "images": [
      {
        "name": "docker",
        "reference": "1.12"
      }
    ]
  }
}

$ cat /run/metadata/torcx     
TORCX_LOWER_PROFILES="vendor"
TORCX_UPPER_PROFILE="systemd"
TORCX_PROFILE_PATH="/run/torcx/profile.json"
TORCX_BINDIR="/run/torcx/bin"
TORCX_UNPACKDIR="/run/torcx/unpack"

$ cat /run/torcx/profile.json 
{
  "kind": "profile-manifest-v0",
  "value": {
    "images": [
      {
        "name": "docker",
        "reference": "1.12"
      }
    ]
  }
}

Note: kube-spawn uses the developer_container image, not the regular production images. I am not sure if that can be the source of any oddities.