`Rpm-ostree updates` sometimes fails on my machine

[kuba3351]

Host system details

Fedora 39 Sericea:

 jsierzega  /var/home/jsierzega  rpm-ostree status
State: busy
Transaction: update 
  Initiator: client(id:cli dbus:1.126 unit:session-1.scope uid:1000)
Deployments:
● fedora:fedora/39/x86_64/sericea
                  Version: 39.20231218.0 (2023-12-18T01:26:36Z)
               BaseCommit: f823242a2faab27a62b89c6614345ca75073398d062196b151318ae93e4db112
             GPGSignature: Valid signature by E8F23996F23218640CB44CBE75CF5AC418B8E74C
      RemovedBasePackages: toolbox 0.0.99.4-5.fc39
          LayeredPackages: 'g++' android-tools ansible bats chromium cloud-utils cmake diffstat distrobox dnf doxygen edk2-ovmf gettext git git-credential-libsecret gparted hdparm htop hyprland kcat kubernetes
                           langpacks-pl libcamera libffi-devel libreoffice libvirt lxterminal maven meson moby-engine mousepad mozilla-openh264 ncdu neofetch ngrep nmap nnn nodejs npm nwg-shell openssl p7zip
                           p7zip-plugins pamixer patch patchutils pip pipewire-plugin-libcamera podman-compose postgresql python-sphinx python-sphinx_rtd_theme python3-bcc python3-devel python3-pip
                           python3-wxpython4 qemu ranger rdesktop remmina speech-dispatcher strace subversion swtpm swtpm-tools syslinux systemtap thunderbird tmux traceroute virt-manager w3m WoeUSB xarchiver
                           xorriso yum-utils
                Initramfs: regenerate

  fedora:fedora/39/x86_64/sericea
                  Version: 39.20231205.0 (2023-12-05T00:54:17Z)
               BaseCommit: 65611298218b8cf9febf7c2bcbd08784d0024265cd0da3afdfda5941aaefa5da
             GPGSignature: Valid signature by E8F23996F23218640CB44CBE75CF5AC418B8E74C
      RemovedBasePackages: toolbox 0.0.99.4-5.fc39
          LayeredPackages: 'g++' android-tools ansible bats chromium cloud-utils cmake diffstat distrobox dnf doxygen edk2-ovmf gettext git git-credential-libsecret gparted hdparm htop hyprland kcat kubernetes
                           langpacks-pl libcamera libffi-devel libreoffice libvirt lxterminal maven meson moby-engine mousepad mozilla-openh264 ncdu neofetch ngrep nmap nnn nodejs npm nwg-shell openssl p7zip
                           p7zip-plugins pamixer patch patchutils pip pipewire-plugin-libcamera podman-compose postgresql python-sphinx python-sphinx_rtd_theme python3-bcc python3-devel python3-pip
                           python3-wxpython4 qemu ranger rdesktop remmina speech-dispatcher strace subversion swtpm swtpm-tools syslinux systemtap thunderbird tmux traceroute virt-manager w3m WoeUSB xarchiver
                           xorriso yum-utils
                Initramfs: regenerate

Expected vs actual behavior

# rpm-ostree update
Applying 1 override and 778 overlays
Processing packages... done
Running pre scripts... done
error: Running %prein for brlapi: bwrap(/bin/sh): Proces potomny został zakończony sygnałem 10; run `journalctl -t 'rpm-ostree(brlapi.prein)'` for more information

 jsierzega  /var/home/jsierzega  130  journalctl -t 'rpm-ostree(brlapi.prein)'
gru 15 16:30:51 NBJSIERZEGA rpm-ostree(brlapi.prein)[146851]: groupadd.rpmostreesave: /etc/group.4: lock file already used (nlink: 1)
gru 15 16:30:51 NBJSIERZEGA rpm-ostree(brlapi.prein)[146851]: groupadd.rpmostreesave: cannot lock /etc/group; try again later.
-- Boot 16861351a5d744fa9c60f842b8e97bf3 --
gru 19 08:52:15 NBJSIERZEGA rpm-ostree(brlapi.prein)[20147]: groupadd.rpmostreesave: /etc/group.4: lock file already used (nlink: 1)
gru 19 08:52:15 NBJSIERZEGA rpm-ostree(brlapi.prein)[20147]: groupadd.rpmostreesave: cannot lock /etc/group; try again later.

Expected:

# rpm-ostree upgrade
...
Success!

Would you like to work on the issue?
No, I don't have necessary knowledge

Additional information

The issue occurs only sometimes, If i execute rpm-ostree update again, it success

[COM8]

The pre install script for brlapi fails. Probably they are doing something not compatible with ostree/a readonly OS.

What's the output of journalctl -t 'rpm-ostree(brlapi.prein)?

[kuba3351]

Hello

I already attached output of this command in my first post.

And I have a question: If this script is doing something unsupported, why the update sometimes ending with success? Usually after I boot up my system, the first time fails but the second time completes successfully.

[COM8]

Oh, sorry for that!
I had a quick look at the spec file definition for the package. and they are not using systemd-sysusers for adding the user (yet?). This can lead to race conditions like you are encountering here.

I'd suggest opening an issue over at the brlapi issue tracker and asking them to switch to systemd-sysusers.

But this cures only the triggering symptom. The real question is what is causing the race condition?

Are you able to add new groups on your own? e.g. groupadd -r test_group

[kuba3351]

Yes, I can but I have to use sudo or be root.

After adding I can find the new group in /etc/group

[kuba3351]

Thank you for investigation.

Created an issue: https://bugzilla.redhat.com/show_bug.cgi?id=2255377

[kuba3351]

It seems that they resolve the issue, I didn't seen this error for a long time, so closing it.