cockpit-project/cockpit-machines

updates-testing: virt-ssh-helper: could not proxy traffic: internal error: EOF on stdin: Connection reset by peer

Closed this issue · 15 comments

Duplicate of #1739

Actually no, this is something different now. The screenshot says

Migration failed
Cannot recv data: Warning: Permanently added 'machine2' (ED25519) to the list of known hosts. virt-ssh-helper: could not proxy traffic: internal error: EOF on stdin: Connection reset by peer

It happened yesterday in #1742 first. But #1741 it was still the other SELinux related breakage. So this is urgent, the update didn't land yet.

@mvollmer @jelly I'm still out this week, so someone else needs to do some piloting, sorry. Can you please have a look? I hope we can still block this in updates-testing.

The image diff is:

Changed:
  NetworkManager-openvpn (1:1.12.0-1.fc40 -> 1:1.10.2-6.fc40)
  ansible-core (2.16.9-1.fc40 -> 2.16.8-1.fc40)
  btrfs-progs (6.9.2-1.fc40 -> 6.8.1-1.fc40)
  conmon (2:2.1.12-1.fc40 -> 2:2.1.10-1.fc40)
  criu (3.19-6.fc40 -> 3.19-4.fc40)
  criu-libs (3.19-6.fc40 -> 3.19-4.fc40)
  crypto-policies (20240725-1.git28d3e2d.fc40 -> 20240510-1.gitd287a42.fc40)
  crypto-policies-scripts (20240725-1.git28d3e2d.fc40 -> 20240510-1.gitd287a42.fc40)
  curl (8.6.0-9.fc40 -> 8.6.0-8.fc40)
  dracut (102-2.fc40 -> 101-1.fc40)
  dracut-config-generic (102-2.fc40 -> 101-1.fc40)
  dracut-network (102-2.fc40 -> 101-1.fc40)
  dracut-squash (102-2.fc40 -> 101-1.fc40)
  flashrom (1.4.0-1.fc40 -> 1.3.0-6.fc40)
  fwupd (1.9.22-1.fc40 -> 1.9.21-1.fc40)
  fwupd-plugin-flashrom (1.9.22-1.fc40 -> 1.9.21-1.fc40)
  fwupd-plugin-modem-manager (1.9.22-1.fc40 -> 1.9.21-1.fc40)
  fwupd-plugin-uefi-capsule-data (1.9.22-1.fc40 -> 1.9.21-1.fc40)
  gettext (0.22.5-4.fc40 -> 0.22.5-2.fc40)
  gettext-envsubst (0.22.5-4.fc40 -> 0.22.5-2.fc40)
  gettext-libs (0.22.5-4.fc40 -> 0.22.5-2.fc40)
  gettext-runtime (0.22.5-4.fc40 -> 0.22.5-2.fc40)
  gsettings-desktop-schemas (46.1-1.fc40 -> 46.0-1.fc40)
  gvisor-tap-vsock (6:0.7.4-1.fc40 -> 6:0.7.3-2.fc40)
  gvisor-tap-vsock-gvforwarder (6:0.7.4-1.fc40 -> 6:0.7.3-2.fc40)
  hwdata (0.384-1.fc40 -> 0.383-1.fc40)
  kdump-utils (1.0.44-1.fc40 -> 1.0.42-10.fc40)
  kernel-core (6.9.12-200.fc40 -> 6.9.9-200.fc40)
  kernel-modules-core (6.9.12-200.fc40 -> 6.9.9-200.fc40)
  kernel-tools (6.9.12-200.fc40 -> 6.9.9-200.fc40)
  kernel-tools-libs (6.9.12-200.fc40 -> 6.9.9-200.fc40)
  kexec-tools (2.0.28-12.fc40 -> 2.0.28-10.fc40)
  less (643-5.fc40 -> 643-4.fc40)
  libarchive (3.7.2-6.fc40 -> 3.7.2-4.fc40)
  libcurl (8.6.0-9.fc40 -> 8.6.0-8.fc40)
  libdrm (2.4.122-1.fc40 -> 2.4.121-1.fc40)
  libnl3 (3.10.0-1.fc40 -> 3.9.0-3.fc40)
  libnl3-cli (3.10.0-1.fc40 -> 3.9.0-3.fc40)
  libtirpc (1.3.5-0.fc40 -> 1.3.4-1.rc3.fc40)
  linux-system-roles (1.84.0-1.fc40 -> 1.82.0-1.fc40)
  mesa-dri-drivers (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  mesa-filesystem (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  mesa-libEGL (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  mesa-libGL (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  mesa-libgbm (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  mesa-libglapi (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  mesa-va-drivers (24.1.4-3.fc40 -> 24.1.2-8.fc40)
  nbdkit (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nbdkit-basic-filters (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nbdkit-basic-plugins (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nbdkit-curl-plugin (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nbdkit-selinux (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nbdkit-server (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nbdkit-ssh-plugin (1.38.3-1.fc40 -> 1.38.2-1.fc40)
  nginx-mimetypes (2.1.54-6.fc40 -> 2.1.54-5.fc40)
  oddjob (0.34.7-13.fc40 -> 0.34.7-12.fc40)
  oddjob-mkhomedir (0.34.7-13.fc40 -> 0.34.7-12.fc40)
  openvpn (2.6.12-1.fc40 -> 2.6.11-1.fc40)
  passt (0^20240726.g57a21d2-1.fc40 -> 0^20240624.g1ee2eca-1.fc40)
  passt-selinux (0^20240726.g57a21d2-1.fc40 -> 0^20240624.g1ee2eca-1.fc40)
  pciutils-libs (3.13.0-1.fc40 -> 3.12.0-1.fc40)
  podman (5:5.2.0~rc2-1.fc40 -> 5:5.1.1-1.fc40)
  python3-boto3 (1.34.149-1.fc40 -> 1.34.141-1.fc40)
  python3-botocore (1.34.149-1.fc40 -> 1.34.141-1.fc40)
  python3-perf (6.9.12-200.fc40 -> 6.9.9-200.fc40)
  python3-setuptools (69.0.3-4.fc40 -> 69.0.3-3.fc40)
  qt5-srpm-macros (5.15.14-2.fc40 -> 5.15.14-1.fc40)
  qt6-srpm-macros (6.7.2-2.fc40 -> 6.7.2-1.fc40)
  redhat-rpm-config (288-1.fc40 -> 286-1.fc40)
  rpcbind (1.2.7-0.fc40 -> 1.2.6-4.rc3.fc40)
  rpm-sequoia (1.7.0-1.fc40 -> 1.6.0-3.fc40)
  selinux-policy (40.26-1.fc40 -> 40.23-1.fc40)
  selinux-policy-devel (40.26-1.fc40 -> 40.23-1.fc40)
  selinux-policy-targeted (40.26-1.fc40 -> 40.23-1.fc40)
  strace (6.10-1.fc40 -> 6.9-1.fc40)
  systemd (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-container (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-libs (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-networkd (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-oomd-defaults (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-pam (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-resolved (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-rpm-macros (255.10-1.fc40 -> 255.8-1.fc40)
  systemd-udev (255.10-1.fc40 -> 255.8-1.fc40)
  tuned (2.24.0-0.1.rc1.fc40 -> 2.23.0-5.fc40)
  valkey (7.2.5-9.fc40 -> 7.2.5-8.fc40)
  xen-libs (4.18.2-4.fc40 -> 4.18.2-1.fc40)
  xen-licenses (4.18.2-4.fc40 -> 4.18.2-1.fc40)
  zchunk-libs (1.5.1-1.fc40 -> 1.4.0-2.fc40)

Still some selinux violations in logs:

Jul 29 09:19:52 fedora-40-127-0-0-2-2201 setroubleshoot[2722]: SELinux is preventing rpc-virtqemud from read access on the blk_file sda. For complete SELinux messages run: sealert -l dae1f9a8-97f0-47f3-8496-9606a5047880
Jul 29 09:19:52 fedora-40-127-0-0-2-2201 setroubleshoot[2722]: SELinux is preventing rpc-virtqemud from read access on the blk_file sda.

                                                               *****  Plugin catchall (100. confidence) suggests   **************************

                                                               If you believe that rpc-virtqemud should be allowed read access on the sda blk_file by default.
                                                               Then you should report this as a bug.
                                                               You can generate a local policy module to allow this access.
                                                               Do
                                                               allow this access for now by executing:
                                                               # ausearch -c 'rpc-virtqemud' --raw | audit2allow -M my-rpcvirtqemud
                                                               # semodule -X 300 -i my-rpcvirtqemud.pp

@jelly ah, nice -- I grepped for something like permissive=0 and didn't find much, but perhaps that was eaten up by setroubleshoot? Does it work with setenforce 0?

I tried setenforce 0 but that wasn't helping so instead did:

ausearch -c 'rpc-virtqemud' --raw | audit2allow -M my-rpcvirtqemud
semodule -i my-rpcvirtqemud.pp

ausearch -c 'virtqemud' --raw | audit2allow -M my-virtqemud
semodule -X 300 -i my-virtqemud.pp

journalctl shows no more violations but it is still broken..

Nevermind, setenforce 0 and clicking the submit button again lets the test pass.

So I suppose this requires another bugzilla

@jelly WDYM with the "image diff" here? This isn't an image refresh, and all the versions are going down. But indeed the test run here shows that e.g. selinux-policy got updated to 40.26-1.fc40. Curious..

So that is a regression from https://bodhi.fedoraproject.org/updates/FEDORA-2024-f6d12d5c36 ? Or rather "was", as it went into stable now.

@jelly WDYM with the "image diff" here? This isn't an image refresh, and all the versions are going down. But indeed the test run here shows that e.g. selinux-policy got updated to 40.26-1.fc40. Curious..

Maybe that is because I put the machine with updates-testing enabled first and our fedora-40 base image last

So that is a regression from https://bodhi.fedoraproject.org/updates/FEDORA-2024-f6d12d5c36 ? Or rather "was", as it went into stable now.

Aha, likely. I can re-create an image and only pull that package in to verify

sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-f6d12d5c36 confirms. So bugzilla it is.

I'll continue this.

Works with selinux-policy-40.23-1.fc40.noarch. Fails with 40.26-1.fc40. (dnf update selinux-policy). Indeed journalctl -b | grep denied shows several denials, but they are all permissive=1. journalctl -b | grep permissive=0 is empty. I tried

sed -i '/-a task,never/d' /etc/audit/rules.d/audit.rules
service auditd restart

but that doesn't give anything new either.

Reported to https://bugzilla.redhat.com/show_bug.cgi?id=2301910 , created naughty at cockpit-project/bots#6678.

Sent cockpit-project/bots#6679