test-kitchen/kitchen-docker

kitchen create with systemd as run_command affects OS session

bogn83 opened this issue · 4 comments

On ubuntu having run_command: /lib/systemd/systemd in kitchen.yml and calling kitchen test or kitchen create will spawn a new systemd "session".

At first I thought I was being logged out and on some runs graphics got messed up completely. But then I found out that switching through Linux' different tty terminals (Ctrl+Alt+F1-F7) I could actually get back to my graphical session with everything still running.

Currently this setup is not usable I would say:
kubuntu 18.04
systemd 237 with /lib/systemd/systemd as run_command
kitchen-docker 2.9.0
kitchen-salt 0.6.0
kitchen-sync 2.2.1
test-kitchen 2.0.1

I was thinking maybe those are recent incompatibilities between the kitchen-* gems and switched to the versions that were recent when the article I was following was published (Jan 7). But these didn't help either:

gem 'kitchen-salt', '0.4.0'                                                                                                                                                                                                                                                                                                                                           
gem 'kitchen-docker', '2.7.0'                                                                                                                                                                                                                                                                                                                                         
gem 'kitchen-sync', '2.2.1'

I'm running into this as well trying to run an Ubuntu container on Arch.

My kitchen.yml:

platforms:
  - name: ubuntu-18.04
    # Special config since we need to start systemd
    driver_config:
      run_command: /sbin/init
      privileged: true
      provision_command:
        - sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
        - systemctl enable ssh.service
      forward: 443
  • kitchen-docker (2.8.0)
  • test-kitchen (1.24.0)

In my case, I can't even get back to my x11 session; it disconnects and the system becomes unusable. I'm thinking this might be an issue with docker specifically, but I don't know how to dig further. Is there a way to see exactly what dockerfile and docker command docker-kitchen is generating?

I found a solution! This behavior was also reported upstream on Docker: docker/for-linux#106 (comment)

The TL;DR is: don't use privileged: true, which is what a ton of guides on the internet say to do. Use cap_add: - SYS_ADMIN. Taking the info I linked above, my working kitchen.yml looks like this:

platforms:
  - name: ubuntu-18.04
    # Special config since we need to start systemd
    driver_config:
      run_command: /sbin/init
      cap_add:
        - SYS_ADMIN
      run_options:
        env: container=docker
      volume:
        - /sys/fs/cgroup:/sys/fs/cgroup
      provision_command:
        - sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
        - systemctl enable ssh.service

I don't know if that env: container=docker part is necessary, but the other person brought it up and it works.

Based on the link above, I found https://github.com/systemd/systemd/blob/aa0c34279ee40bce2f9681b496922dedbadfca19/src/basic/virt.c#L434 and the ubuntu Dockerfile (https://github.com/tianon/docker-brew-ubuntu-core/blob/bcdd3818e932eda677301ae5d81c80e010793c91/bionic/Dockerfile) which made me need to add mkdir -p /run/systemd && echo 'docker' > /run/systemd/container to provision_command to get that setup working with a stock Debian image

Im following exactly the Saltstack Formula test platforms and I get this issue too, tried the options above and no luck so far.