ostreedev/ostree-rs-ext

`rpm-ostreed` refuses to start after a rebase from Fedora 35 -> 36

anthr76 opened this issue · 6 comments

I'm currently building a well mutated tree for sway WM for myself here.

I was running this for a couple months on Fedora 35. Yesterday I decided to rebase to Fedora 36 via rpm-ostree rebase --experimental ostree-unverified-registry:ghcr.io/anthr76/sway-desktop:36

This all goes smoothly but once booted into a Fedora 36 entry rpm-ostree refuses to work for:

"error: Couldn't start daemon: Error setting up sysroot: Reading deployment 0: Loading origin status: Invalid refspec ostree-unverified-registry:ghcr.io/anthr76/sway-desktop:36"

I was able to boot into a pinned Fedora 35 entry and do a 'upgrade' and that works. How can I debug this further to provide more useful information?

lucab commented

Thanks for the report. Can you please note down here what are the version of the rpm-ostree package in the initial and in the target/failing deployments?

Sure:

On Fedora 35:

ignoring ``: invalid filter directive
rpm-ostree:
 Version: '2022.6'
 Git: cbbf42f83757288b9d7dc4415073623cdc367cf4
 Features:
  - rust
  - compose
  - fedora-integration

On Fedora 36:

rpm-ostree-2022.8-1

I reckon 2022.6 may be broken?

I just tried this deployment though rpm-ostree didn't start

  ostree-unverified-registry:ghcr.io/anthr76/sway-desktop:35
                    Digest: sha256:9223abcc971b5c1804f97606d45e74d4108a4a57bd0fd360e481b70edea0ad53
                   Version: 35.2022-05-07T00:21:14.1.0 (2022-05-07T00:32:14Z)
                    Commit: 08c0bcbdf05fa9d27b0b93c1f88f471b40273a28aac6e339ee3fb29b56ee17c4
                            ├─ fedora-35-updates (2022-05-05T07:02:20Z)
                            ├─ fedora-35-updates-testing (2022-05-05T06:38:38Z)
                            └─ fedora-35 (2021-10-26T05:31:27Z)
                    Staged: no
                 StateRoot: fedora
                  Upgraded: SDL2 2.0.20-1.fc35 -> 2.0.22-2.fc35
                            bind-libs 32:9.16.27-1.fc35 -> 32:9.16.28-1.fc35
                            bind-license 32:9.16.27-1.fc35 -> 32:9.16.28-1.fc35
                            bind-utils 32:9.16.27-1.fc35 -> 32:9.16.28-1.fc35
                            buildah 1.23.3-2.fc35 -> 1.23.3-3.fc35
                            cifs-utils 6.13-3.fc35 -> 6.15-1.fc35
                            cifs-utils-info 6.13-3.fc35 -> 6.15-1.fc35
                            container-selinux 2:2.181.0-1.fc35 -> 2:2.183.0-3.fc35
                            cpp 11.2.1-10.fc35 -> 11.3.1-2.fc35
                            cups-filters 1.28.14-1.fc35 -> 1.28.15-1.fc35
                            cups-filters-braille 1.28.14-1.fc35 -> 1.28.15-1.fc35
                            cups-filters-libs 1.28.14-1.fc35 -> 1.28.15-1.fc35
                            curl 7.79.1-1.fc35 -> 7.79.1-3.fc35
                            ddccontrol-db 20220406-1.fc35 -> 20220414-1.fc35
                            dhcp-client 12:4.4.2-16.b1.fc35 -> 12:4.4.3-2.fc35
                            dhcp-common 12:4.4.2-16.b1.fc35 -> 12:4.4.3-2.fc35
                            evolution-data-server 3.42.4-2.fc35 -> 3.42.5-1.fc35
                            evolution-data-server-langpacks 3.42.4-2.fc35 -> 3.42.5-1.fc35
                            fedora-chromium-config 1.1-12.fc35 -> 1.2-3.fc35
                            firefox 99.0-1.fc35 -> 100.0-2.fc35
                            flatpak 1.12.7-1.fc35 -> 1.12.7-2.fc35
                            flatpak-libs 1.12.7-1.fc35 -> 1.12.7-2.fc35
                            flatpak-selinux 1.12.7-1.fc35 -> 1.12.7-2.fc35
                            flatpak-session-helper 1.12.7-1.fc35 -> 1.12.7-2.fc35
                            freerdp-libs 2:2.6.1-1.fc35 -> 2:2.7.0-1.fc35
                            freetype 2.11.0-3.fc35 -> 2.11.0-6.fc35
                            fwupd-efi 1.2-1.fc35 -> 1.3-1.fc35
                            gcc 11.2.1-10.fc35 -> 11.3.1-2.fc35
                            gdisk 1.0.8-2.fc35 -> 1.0.9-1.fc35
                            git 2.35.1-1.fc35 -> 2.35.3-1.fc35
                            git-core 2.35.1-1.fc35 -> 2.35.3-1.fc35
                            git-core-doc 2.35.1-1.fc35 -> 2.35.3-1.fc35
                            glibc 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-all-langpacks 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-common 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-devel 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-gconv-extra 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-headers-x86 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-langpack-de 2.34-29.fc35 -> 2.34-32.fc35
                            glibc-langpack-en 2.34-29.fc35 -> 2.34-32.fc35
                            glibmm24 2.66.2-1.fc35 -> 2.66.4-1.fc35
                            gnome-desktop3 41.5-1.fc35 -> 41.6-1.fc35
                            gnutls 3.7.2-3.fc35 -> 3.7.4-1.fc35
                            gnutls-dane 3.7.2-3.fc35 -> 3.7.4-1.fc35
                            gnutls-utils 3.7.2-3.fc35 -> 3.7.4-1.fc35
                            gtkmm30 3.24.5-2.fc35 -> 3.24.6-1.fc35
                            gzip 1.10-5.fc35 -> 1.10-6.fc35
                            hwdata 0.358-1.fc35 -> 0.359-1.fc35
                            inih 49-4.fc35 -> 55-1.fc35
                            iwl100-firmware 39.31.5.1-130.fc35 -> 39.31.5.1-131.fc35
                            iwl1000-firmware 1:39.31.5.1-130.fc35 -> 1:39.31.5.1-131.fc35
                            iwl105-firmware 18.168.6.1-130.fc35 -> 18.168.6.1-131.fc35
                            iwl135-firmware 18.168.6.1-130.fc35 -> 18.168.6.1-131.fc35
                            iwl2000-firmware 18.168.6.1-130.fc35 -> 18.168.6.1-131.fc35
                            iwl2030-firmware 18.168.6.1-130.fc35 -> 18.168.6.1-131.fc35
                            iwl3160-firmware 1:25.30.13.0-130.fc35 -> 1:25.30.13.0-131.fc35
                            iwl3945-firmware 15.32.2.9-130.fc35 -> 15.32.2.9-131.fc35
                            iwl4965-firmware 228.61.2.24-130.fc35 -> 228.61.2.24-131.fc35
                            iwl5000-firmware 8.83.5.1_1-130.fc35 -> 8.83.5.1_1-131.fc35
                            iwl5150-firmware 8.24.2.2-130.fc35 -> 8.24.2.2-131.fc35
                            iwl6000-firmware 9.221.4.1-130.fc35 -> 9.221.4.1-131.fc35
                            iwl6000g2a-firmware 18.168.6.1-130.fc35 -> 18.168.6.1-131.fc35
                            iwl6000g2b-firmware 18.168.6.1-130.fc35 -> 18.168.6.1-131.fc35
                            iwl6050-firmware 41.28.5.1-130.fc35 -> 41.28.5.1-131.fc35
                            iwl7260-firmware 1:25.30.13.0-130.fc35 -> 1:25.30.13.0-131.fc35
                            iwlax2xx-firmware 20220310-130.fc35 -> 20220411-131.fc35
                            kernel 5.16.19-200.fc35 -> 5.17.5-200.fc35
                            kernel-core 5.16.19-200.fc35 -> 5.17.5-200.fc35
                            kernel-devel 5.16.19-200.fc35 -> 5.17.5-200.fc35
                            kernel-headers 5.16.5-200.fc35 -> 5.17.4-200.fc35
                            kernel-modules 5.16.19-200.fc35 -> 5.17.5-200.fc35
                            kernel-modules-extra 5.16.19-200.fc35 -> 5.17.5-200.fc35
                            langtable 0.0.57-1.fc35 -> 0.0.58-1.fc35
                            libcurl 7.79.1-1.fc35 -> 7.79.1-3.fc35
                            libertas-sd8686-firmware 20220310-130.fc35 -> 20220411-131.fc35
                            libertas-sd8787-firmware 20220310-130.fc35 -> 20220411-131.fc35
                            libertas-usb8388-firmware 2:20220310-130.fc35 -> 2:20220411-131.fc35
                            libgcc 11.2.1-10.fc35 -> 11.3.1-2.fc35
                            libgomp 11.2.1-10.fc35 -> 11.3.1-2.fc35
                            libhandy 1.4.0-1.fc35 -> 1.4.1-1.fc35
                            libinput 1.19.3-1.fc35 -> 1.19.4-1.fc35
                            libipa_hbac 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            libsmbclient 2:4.15.6-0.fc35 -> 2:4.15.7-0.fc35
                            libsolv 0.7.21-1.fc35 -> 0.7.22-1.fc35
                            libsss_certmap 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            libsss_idmap 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            libsss_nss_idmap 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            libsss_sudo 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            libstdc++ 11.2.1-10.fc35 -> 11.3.1-2.fc35
                            libtpms 0.9.2-0.20220106gite81d634c27.fc35.0 -> 0.9.4-0.20220425gite4d68670e1.fc35.0
                            libwbclient 2:4.15.6-0.fc35 -> 2:4.15.7-0.fc35
                            libwinpr 2:2.6.1-1.fc35 -> 2:2.7.0-1.fc35
                            libxml2 2.9.13-1.fc35 -> 2.9.14-1.fc35
                            linux-firmware 20220310-130.fc35 -> 20220411-131.fc35
                            linux-firmware-whence 20220310-130.fc35 -> 20220411-131.fc35
                            oniguruma 6.9.7.1-1.fc35.1 -> 6.9.8-1.fc35
                            perl-Git 2.35.1-1.fc35 -> 2.35.3-1.fc35
                            pipewire 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            pipewire-alsa 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            pipewire-gstreamer 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            pipewire-jack-audio-connection-kit 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            pipewire-libs 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            pipewire-pulseaudio 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            pipewire-utils 0.3.50-1.fc35 -> 0.3.51-1.fc35
                            podman 3:3.4.4-1.fc35 -> 3:3.4.7-1.fc35
                            podman-gvproxy 3:3.4.4-1.fc35 -> 3:3.4.7-1.fc35
                            podman-plugins 3:3.4.4-1.fc35 -> 3:3.4.7-1.fc35
                            python3-libxml2 2.9.13-1.fc35 -> 2.9.14-1.fc35
                            python3-requests 2.27.0-1.fc35 -> 2.27.1-2.fc35
                            qt5-qtwayland 5.15.2-21.fc35 -> 5.15.2-22.fc35
                            rpm-ostree 2022.6-2.fc35 -> 2022.8-1.fc35
                            rpm-ostree-libs 2022.6-2.fc35 -> 2022.8-1.fc35
                            rsync 3.2.3-8.fc35 -> 3.2.3-9.fc35
                            rtkit 0.11-28.fc35 -> 0.11-30.fc35
                            samba-client 2:4.15.6-0.fc35 -> 2:4.15.7-0.fc35
                            samba-client-libs 2:4.15.6-0.fc35 -> 2:4.15.7-0.fc35
                            samba-common 2:4.15.6-0.fc35 -> 2:4.15.7-0.fc35
                            samba-common-libs 2:4.15.6-0.fc35 -> 2:4.15.7-0.fc35
                            sssd 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-ad 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-client 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-common 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-common-pac 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-ipa 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-kcm 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-krb5 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-krb5-common 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-ldap 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-nfs-idmap 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            sssd-proxy 2.6.3-1.fc35 -> 2.7.0-1.fc35
                            systemd 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-container 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-libs 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-networkd 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-oomd-defaults 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-pam 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-resolved 249.11-2.fc35 -> 249.12-3.fc35
                            systemd-udev 249.11-2.fc35 -> 249.12-3.fc35
                            vim-common 2:8.2.4701-1.fc35 -> 2:8.2.4845-1.fc35
                            vim-data 2:8.2.4701-1.fc35 -> 2:8.2.4845-1.fc35
                            vim-enhanced 2:8.2.4701-1.fc35 -> 2:8.2.4845-1.fc35
                            vim-filesystem 2:8.2.4701-1.fc35 -> 2:8.2.4845-1.fc35
                            vim-minimal 2:8.2.4701-1.fc35 -> 2:8.2.4845-1.fc35
                            virtualbox-guest-additions 6.1.32-1.fc35 -> 6.1.34-1.fc35
                            webkit2gtk3 2.36.0-2.fc35 -> 2.36.1-1.fc35
                            webkit2gtk3-jsc 2.36.0-2.fc35 -> 2.36.1-1.fc35
                            xdg-desktop-portal 1.12.3-1.fc35 -> 1.12.4-1.fc35
                            xz 5.2.5-7.fc35 -> 5.2.5-9.fc35
                            xz-libs 5.2.5-7.fc35 -> 5.2.5-9.fc35
                            zchunk-libs 1.2.1-1.fc35 -> 1.2.2-1.fc35
                     Added: libjose-11-3.fc35.x86_64
                            sssd-idp-2.7.0-1.fc35.x86_64

Invalid refspec ostree-unverified-registry:ghcr.io/anthr76/sway-desktop:36"

It looks like you're using the experimental ostree-native-container flow. We made some incompatible changes to this recently in the backend. I think we may consider auto-adjusting if we get more reports of this, but for now you can fix this by doing e.g.:

$ unshare -m
# mount -o remount,rw /sysroot
# vi /ostree/deploy/$stateroot/deploy/$checksum.origin

(Replace $stateroot and $checksum with your specific values)

And make the top look like this:

[origin]
container-image-reference=ostree-unverified-registry:ghcr.io/anthr76/sway-desktop:36

Thanks @cgwalters that did the trick. Maybe an announcement was made more clearly but I've been a little busy.

If you do feel it wasn't shouted enough for the breaking change I can throw something in documentation?

Yes if you like; it'd be a change in the rpm-ostree docs. The relevant code was actually never in this repo, only there.

For now I'm closing this, but without prejudice - if we get more reports of this we may look at trying to auto-adjust.