error: Committing: Adding composefs metadata: No data available
Closed this issue · 12 comments
Host system details
State: idle
Deployments:
● fedora:fedora/38/x86_64/kinoite
Version: 38.20231009.0 (2023-10-09T01:14:47Z)
BaseCommit: c86269550b6843d4a7e2b27f8ced25079abf6391c07653c47753e5e6be7742d8
GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464
RemovedBasePackages: firefox firefox-langpacks 118.0.1-7.fc38 plasma-discover-rpm-ostree plasma-discover plasma-discover-notifier plasma-discover-flatpak 5.27.8-1.fc38
LayeredPackages: conda conntrack-tools dejavu-fonts-all fzf git gnu-free-mono-fonts ksshaskpass libvirt libvirt-daemon-kvm lm_sensors virt-manager waydroid wireguard-tools
fedora:fedora/38/x86_64/kinoite
Version: 38.20231012.0 (2023-10-12T01:02:34Z)
Commit: 77b5fb9cfa654d48839617a237798fa9c2cd91d31e5aa4a86f48f910abd526c4
GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464Expected vs actual behavior
Cannot do anything other than rpm-ostree reset. Upgrading, removing packages, installing new packages does not work. After rpm-ostree reset upgrading works, but changing the packages still does not.
rpm-ostree remove waydroid
Inactive requests:
podman (already provided by podman-5:4.7.0-1.fc38.x86_64)
nftables (already provided by nftables-1:1.0.5-2.fc38.x86_64)
Checking out tree c862695... done
Resolving dependencies... done
Applying 6 overrides and 197 overlays
Processing packages... done
Running pre scripts... done
Running post scripts... done
Running posttrans scripts... done
Writing rpmdb... done
Writing OSTree commit... done
error: Committing: Adding composefs metadata: No data available
Expected:
# rpm-ostree remove waydroid
.....
Success!
Steps to reproduce it
Would you like to work on the issue?
Can only provide more info about the setup.
Hmm, must be a regression from #4594
Downgrading to the previous rpm-ostree should fix it, can you confirm?
"No data available" is odd, will look.
What filesystem are you using? i.e. paste the output of stat -f /
~ # stat -f /
File: "/"
ID: 74b629b43ff7215b Namelen: 255 Type: btrfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 249630976 Free: 116566620 Available: 115966172
Inodes: Total: 0 Free: 0
I also have another machine that was following the upgrades similarly, and have the same additional packages installed, but itdoes not have any default packages removed. On that machine upgrade works.
Damn, I forgot before, but I need to add that lately (cannot tell if after latest successfull update or before) I have canceled the ongoing upgrade (very early in the process) with Ctrl+c
I am not totally sure where this error is coming from, but the latest git ostree (soon to show up in https://copr.fedorainfracloud.org/coprs/g/CoreOS/continuous/ will help debug - want to try installing that via usroverlay?)
Is this reproducible 100% of the time?
The error is happening 100% of the time, but I was able to do rpm-ostree reset and then rpm-ostree upgrade, but even then in the upgraded system I could not uninstall or add new package, so i did rollback (which also worked) to not lose the layered packages which I need. When I tried to deploy older commit it also did not work, but have not tested it together with rpm-ostree reset yet.
Tommorow I might do more tests, including the debug version of rpm-ostree.
I found out that:
~ # ostree fsck -a
Validating refs...
Validating refs in collections...
Enumerating commits...
Verifying content integrity of 213 commit objects...
fsck objects (31035/155646) [== ] 19%In commits ba4338e09c76b76d46bcb8c970f4a3f46acdcae2a3c32c03ffb70ff423795477, b3fb087d0c2103f9296f575ab1f9914bab4f0e7ce8422d42e3fe536f4ab72753, c86269550b6843d4a7e2b27f8ced25079abf6391c07653c47753e5e6be7742d8: fsck content object 1677f333fd976173d257f3983aa96408302f1b0d01fdeff32a5938deb1acba84: Corrupted file object; checksum expected='1677f333fd976173d257f3983aa96408302f1b0d01fdeff32a5938deb1acba84' actual='73a00af44ecb0706519c27968e7dbbef09a8bc2073e6679341dcf07c38215102'
fsck objects (56318/155646) [==== ] 36%In commits ba4338e09c76b76d46bcb8c970f4a3f46acdcae2a3c32c03ffb70ff423795477, b3fb087d0c2103f9296f575ab1f9914bab4f0e7ce8422d42e3fe536f4ab72753, c86269550b6843d4a7e2b27f8ced25079abf6391c07653c47753e5e6be7742d8: fsck content object 5ffff84f608ab26386b4d883aeba1ac0bfce1aa7f029fe7ec80a9329afea8c54: Corrupted file object; checksum expected='5ffff84f608ab26386b4d883aeba1ac0bfce1aa7f029fe7ec80a9329afea8c54' actual='18f65042bd148628cec694a80de5e4c025e1885c1ec39c15cb24355210cd721c'
fsck objects (126067/155646) [========== ] 80%In commits ba4338e09c76b76d46bcb8c970f4a3f46acdcae2a3c32c03ffb70ff423795477, b3fb087d0c2103f9296f575ab1f9914bab4f0e7ce8422d42e3fe536f4ab72753, c86269550b6843d4a7e2b27f8ced25079abf6391c07653c47753e5e6be7742d8: fsck content object 73da4c33ba83d5c53de84a82d95d0c39615aa378c907fdbc729f5f7dc7365694: Error reading from file descriptor: Input/output error
fsck objects (155646/155646) [=============] 100%
In the end I managed to fix the system with steps:
rpm-ostree resetrpm-ostree rebase <commit from ~2 weeks ago>- reboot
rpm-ostree install <my stuff>rpm-ostree upgrade
and here I am on latest commit of fedora/38/x86_64/kinoite
the problematic commit path is still on my system, since I pinned it.
Not sure if this helps with the mystery (I guess some file/commit was corrupted ), if there is need for some debug info, I could revert temporarily to the problematic deployment, to test stuff, but need info what and how.
Closing for now.
Hi, you likely have filesystem corruption. Check dmesg. You may need to reinstall.
I was checking dmesg before and now, no errors there, btrfs scrub shows no error as well, but btrfsck found something. So this might be thee case.
Same error
error: Committing: Adding composefs metadata: Checking out composefs: Processing 55e418cdba0a2e6968669e95bbd3017d40992572ef15199751e648d3f412da3d: No data available
Same error error: Committing: Adding composefs metadata: Checking out composefs: Processing 55e418cdba0a2e6968669e95bbd3017d40992572ef15199751e648d3f412da3d: No data available
For me it was indeed broken filesystem, but if I was able to fix it with btrfs tools, and then, thanks to rpm-ostree powers I was able to restore system fully, without reinstall, and I am still using the same system installation right now.
Right after that I made small set of tips related to managing the system with ostree/rpm-ostree: https://github.com/Szwendacz99/fedora-immutable-cheatsheet
- I did a cleanup as you explain in your cheatsheet
rpm-ostree cleanup -p -b -r -m - Rebase from fedora:fedora/39/x86_64/kinoite to fedora:fedora/40/x86_64/kinoite
rpm-ostree rebase fedora:fedora/40/x86_64/kinoite
Thanks
I had the same issue and running sudo ostree fsck -a --delete seems to have resolved it for me 👍