RFC: qm.container: network = host - change to private ? macvlan?

[dougsland]

As discussed with @alexlarsson and @rhatdan by default users can access some host network settings via sysctl (showed below) from QM env (please note, NOT possible via nested container) as the default configuration is network=host.

Should we change this options to private as default configuration or even macvlan in qm.container?
Maybe just make a good documentation with examples how to change it and values / behaviors?

The network=host was added to simplify the network layer and make it work for most of initial deployments, on top of that, users can always change the settings and adapt for their needs.

On the other side, we have options in podman like --ipc=host that is kind of similar exposition of host and we believe is not appropriate.

host> podman exec -it qm bash
bash-5.1# ./rung-sysctl-tests
Randomly setting sysctl parameters...
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.tcp_max_syn_backlog = 632
net.core.rmem_max = 25982
net.core.wmem_max = 31861

[alexlarsson]

Lets go with private for now. Its safer as the default, and it is what podman in general uses as the default.

[dougsland]

Changing to private worked as expected:

[root@localhost ~]# cat /usr/share/containers/systemd/qm.container | grep network -i
Network=private
[root@localhost ~]# podman exec -it qm bash
bash-5.1# podman pull alpine:latest
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Writing manifest to image destination
05455a08881ea9cf0e752bc48e61bbd71a34c029bb13df01e40e3e70e0d007bd
bash-5.1# podman images
REPOSITORY                         TAG         IMAGE ID      CREATED       SIZE
registry.fedoraproject.org/fedora  latest      e2f83b188263  32 hours ago  229 MB
docker.io/library/alpine           latest      05455a08881e  3 months ago  7.67 MB
bash-5.1# exit
exit
[root@localhost ~]# cat /etc/os-release
NAME="Automotive Stream Distribution"
VERSION="9"
ID="autosd"
ID_LIKE="rhel fedora centos"
VERSION_ID="9"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Automotive Stream Distribution 9"
ANSI_COLOR="0;31"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:centos:autosd:9"
HOME_URL="https://sig.centos.org/automotive/"
REDHAT_SUPPORT_PRODUCT="Automotive Stream Distribution"
REDHAT_SUPPORT_PRODUCT_VERSION="Automotive Stream Distribution 9"
[root@localhost ~]#

Keep in mind, I had to workaround manually until we get a patch for this one here: #406

[rhatdan]

If QM goes private, then individual containers within the QM can not listen on the host network. But I agree it is better to put them in a private network.

[dougsland]

still investigating, executing a bunch of tests.

[dougsland]

So redeployed my machines and the tests went well.

• Step 1 - Build and run the CentOS Auto VM.

#  git clone https://gitlab.com/CentOS/automotive/sample-images.git
# cd sample-images/osbuild-manifests/
# make cs9-qemu-qmcontainer-regular.x86_64.qcow2 'DEFINES=extra_repos=[{"id":"local","baseurl":"file:///root/rpmbuild/RPMS/noarch"}] extra_rpms=["qm-6.0","vim-enhanced","strace","dnf","gdb","polkit","rsync","python3","openssh-server","openssh-clients"]
 ssh_permit_root_login=true osname="autosd" ssh_permit_password_auth=true'
# ./runvm --nographics ./cs9-qemu-qmcontainer-regular.x86_64.qcow2

• Step 2 - Set SSH to be able to login into the VM and use as SECOND terminal

  - Login into the VM (user: root / pass: password)
  - Set /etc/ssh/sshd_config -> PermitRootLogin yes (systemctl restart sshd)
  - In another shell login:  
    netstat -na | more (look for ports like 2222 or 2223 .... )
    ssh root@127.0.0.1 -p 2222 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null

• Step 3 - Execute the tests.

Lets start with some bash kung-fu to generate random numbers to be used in the tests

[root@localhost ~]# podman exec -it qm bash
bash-5.1#
bash-5.1# generate_random_number() {
    local min=$1
    local max=$2
    echo $((RANDOM % (max - min + 1) + min))
}

Execute these commands in the QM side (the numbers should keep changing randomly, as expected):

# sysctl -w net.ipv4.tcp_max_syn_backlog=$(generate_random_number 128 1024)

net.ipv4.tcp_max_syn_backlog = 219 <-- changed to 219 in the QM partition

# sysctl -w net.ipv4.tcp_max_syn_backlog=$(generate_random_number 128 1024)

net.ipv4.tcp_max_syn_backlog = 167 <-- changed to 167 in the QM partition

# sysctl -w net.ipv4.tcp_max_syn_backlog=$(generate_random_number 128 1024)

net.ipv4.tcp_max_syn_backlog = 812 <-- changed to 812 in the QM partition

In another window, keep executing in the HOST the same command to see if the the value has changed (it should NOT change):

# sysctl net.ipv4.tcp_max_syn_backlog

net.ipv4.tcp_max_syn_backlog = 128 <-- kept 128 in the host

[root@localhost ~]# sysctl net.ipv4.tcp_max_syn_backlog

net.ipv4.tcp_max_syn_backlog = 128 <-- kept 128 in the host

[root@localhost ~]# sysctl net.ipv4.tcp_max_syn_backlog

net.ipv4.tcp_max_syn_backlog = 128 <-- kept 128 in the host

Done!

There is another way of double check this (as learned from @rhatdan), see if the namespace id for net is different from the HOST and QM (it should be different).

HOST

[root@localhost ~]# ls -l /proc/self/ns/net
lrwxrwxrwx. 1 root root 0 May  1 04:33 /proc/self/ns/net -> 'net:[4026531840]'

QM

bash-5.1# ls -l /proc/self/ns/net
lrwxrwxrwx. 1 root root 0 May  1 04:33 /proc/self/ns/net -> 'net:[4026532287]'