buildah complaining about missing systemd user session although it's present

Question

buildah complaining about missing systemd user session although it's present

der-eismann opened this issue 6 months ago · 4 comments

BUG REPORT INFORMATION

Description

We are using podman & buildah on our self-hosted GitHub runners to run and build containers. Since we started this >1 year ago, we always had lingering enabled to have correctly working user sessions for pull secrets and other stuff.
At some point in the past weeks, there must have been a change regarding systemd and cgroups, since we suddenly started to get warnings about a missing user session, remarkably the output always appears multiple times.
It looks like this:

$ /bin/buildah version
  time="2024-04-08T15:05:09+02:00" level=warning msg="The cgroupv2 manager is set to systemd but there is no systemd user session available"
  time="2024-04-08T15:05:09+02:00" level=warning msg="For using systemd, you may need to log in using a user session"
  time="2024-04-08T15:05:09+02:00" level=warning msg="Alternatively, you can enable lingering with: `loginctl enable-linger 1001` (possibly as root)"
  time="2024-04-08T15:05:09+02:00" level=warning msg="Falling back to --cgroup-manager=cgroupfs"
  Version:         1.35.1
  Go Version:      go1.21.7 (Red Hat 1.21.7-1.el9)
  Image Spec:      1.1.0
  Runtime Spec:    1.1.0
  CNI Spec:        1.0.0
  libcni Version:  v1.1.2
  image Version:   5.30.0
  Git Commit:      
  Built:           Tue Mar 19 13:37:29 2024
  OS/Arch:         linux/amd64
  BuildPlatform:   linux/amd64
$ /bin/buildah bud -f /home/github/runner_0/_work/project/Dockerfile --format docker --tls-verify=true --layers=false --ulimit=nofile=262144:262144 -v=/home/github/cache/go:/go/pkg/mod:shared,rw -t registry/image:main /home/github/runner_0/_work/project
time="2024-04-08T15:05:09+02:00" level=warning msg="The cgroupv2 manager is set to systemd but there is no systemd user session available"
time="2024-04-08T15:05:09+02:00" level=warning msg="For using systemd, you may need to log in using a user session"
time="2024-04-08T15:05:09+02:00" level=warning msg="Alternatively, you can enable lingering with: `loginctl enable-linger 1001` (possibly as root)"
time="2024-04-08T15:05:09+02:00" level=warning msg="Falling back to --cgroup-manager=cgroupfs"
time="2024-04-08T15:05:09+02:00" level=warning msg="The cgroupv2 manager is set to systemd but there is no systemd user session available"
time="2024-04-08T15:05:09+02:00" level=warning msg="For using systemd, you may need to log in using a user session"
time="2024-04-08T15:05:09+02:00" level=warning msg="Alternatively, you can enable lingering with: `loginctl enable-linger 1001` (possibly as root)"
time="2024-04-08T15:05:09+02:00" level=warning msg="Falling back to --cgroup-manager=cgroupfs"
[1/2] STEP 1/6: FROM golang:1.22-alpine AS builder
...

The builds are done via a separate non-root user, that doesn't have a user session, only a systemd service for the GitHub agent. But we enabled lingering, I can also confirm this when connecting via SSH and checking as root:

[root@runner ~]# loginctl user-status 1001
github (1001)
           Since: Mon 2024-04-08 14:50:49 CEST; 14min ago
           State: lingering
          Linger: yes
            Unit: user-1001.slice

I could also reproduce the issue on my local Fedora 39 setup.

Steps to reproduce the issue:
On Fedora/RHEL-based systems:

sudo useradd test
sudo -u test buildah version => creates warning
sudo loginctl enable-linger 1001 && sudo loginctl user-status 1001 => lingering enabled
sudo -u test buildah version => warnings still appear

Describe the results you received:
Multiple messages like these - two for buildah build, one for buildah version:

  time="2024-04-08T15:05:09+02:00" level=warning msg="The cgroupv2 manager is set to systemd but there is no systemd user session available"
  time="2024-04-08T15:05:09+02:00" level=warning msg="For using systemd, you may need to log in using a user session"
  time="2024-04-08T15:05:09+02:00" level=warning msg="Alternatively, you can enable lingering with: `loginctl enable-linger 1001` (possibly as root)"
  time="2024-04-08T15:05:09+02:00" level=warning msg="Falling back to --cgroup-manager=cgroupfs"

Describe the results you expected:

None of the warnings

Output of rpm -q buildah or apt list buildah:

buildah-1.35.0-1.fc39.x86_64 (Fedora 39)
buildah-1.35.1-1.el9.x86_64 (CentOS Stream 9)

Output of buildah version:

*Fedora 39:*
$ buildah version
Version:         1.35.0
Go Version:      go1.21.7
Image Spec:      1.1.0
Runtime Spec:    1.1.0
CNI Spec:        1.0.0
libcni Version:  
image Version:   5.30.0
Git Commit:      
Built:           Thu Mar  7 14:20:46 2024
OS/Arch:         linux/amd64
BuildPlatform:   linux/amd64

*CentOS 9 Stream:*
$ buildah version
Version:         1.35.1
Go Version:      go1.21.7 (Red Hat 1.21.7-1.el9)
Image Spec:      1.1.0
Runtime Spec:    1.1.0
CNI Spec:        1.0.0
libcni Version:  v1.1.2
image Version:   5.30.0
Git Commit:      
Built:           Tue Mar 19 12:37:29 2024
OS/Arch:         linux/amd64
BuildPlatform:   linux/amd64

Output of podman version if reporting a podman build issue:

Output of cat /etc/*release:

*Fedora 39:*
$ cat /etc/*release
Fedora release 39 (Thirty Nine)
NAME="Fedora Linux"
VERSION="39 (Workstation Edition)"
ID=fedora
VERSION_ID=39
VERSION_CODENAME=""
PLATFORM_ID="platform:f39"
PRETTY_NAME="Fedora Linux 39 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:39"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f39/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=39
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=39
SUPPORT_END=2024-11-12
VARIANT="Workstation Edition"
VARIANT_ID=workstation
Fedora release 39 (Thirty Nine)
Fedora release 39 (Thirty Nine)

*CentOS 9 Stream:*
$ cat /etc/*release
CentOS Stream release 9
NAME="CentOS Stream"
VERSION="9"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="9"
PLATFORM_ID="platform:el9"
PRETTY_NAME="CentOS Stream 9"
ANSI_COLOR="0;31"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:centos:centos:9"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
CentOS Stream release 9
CentOS Stream release 9

Output of uname -a:

*Fedora 39:*
Linux laptop 6.8.4-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Apr  4 20:45:21 UTC 2024 x86_64 GNU/Linux

*CentOS 9 Stream:*
Linux builder 5.14.0-432.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Mar 20 20:26:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

not relevant

Answer 1 · 2024-05-09T00:10:39.000Z

A friendly reminder that this issue had no activity for 30 days.

Answer 2 · 2024-05-09T06:01:35.000Z

Still an issue

Answer 3 · 2024-05-22T08:22:21.000Z

After diving into the code a bit, it seems to be caused by the detection of the systemd user session. In config.go, the env var DBUS_SESSION_BUS_ADDRESS is used to determine if there is a systemd user session.
In our case, we have self-hosted GitHub Action runners, that are started by a user without an active session, but with lingering enabled. When I execute printenv in a GitHub Actions workflow on our runners, there are a lot of GitHub-related env vars, but no DBUS_SESSION_BUS_ADDRESS, causing the above mentioned warning message. Now I wonder if we should just export this env var everywhere, or if there is a better way to detect a systemd user session.

Regarding my reproduction steps, I guess these aren't valid. If I execute this

$ sudo -u test echo $DBUS_SESSION_BUS_ADDRESS
unix:path=/run/user/1000/bus

I get the path for the user who is executing the command, not the test user.

Answer 4 · 2024-06-22T00:10:57.000Z

A friendly reminder that this issue had no activity for 30 days.