Fails to install cuda with error: opendir(local): No such file or directory

[notnotno]

Describe the bug

Unable to install cuda:

rpm-ostree install cuda
Checking out tree 84c4912... done
Enabled rpm-md repositories: rpmfusion-free-updates-testing copr:copr.fedorainfracloud.org:ublue-os:akmods rpmfusion-nonfree-updates cuda-rhel9-x86_64 rpmfusion-free rpmfusion-nonfree-updates-testing cuda-fedora39-x86_64 fedora rpmfusion-free-updates copr:copr.fedorainfracloud.org:principis:howdy rpmfusion-nonfree updates enpass updates-archive
Updating metadata for 'rpmfusion-free-updates-testing'... done
Updating metadata for 'copr:copr.fedorainfracloud.org:ublue-os:akmods'... done
Updating metadata for 'rpmfusion-nonfree-updates'... done
Updating metadata for 'cuda-rhel9-x86_64'... done
Updating metadata for 'rpmfusion-free'... done
Updating metadata for 'rpmfusion-nonfree-updates-testing'... done
Updating metadata for 'cuda-fedora39-x86_64'... done
Updating metadata for 'fedora'... done
Updating metadata for 'rpmfusion-free-updates'... done
Updating metadata for 'copr:copr.fedorainfracloud.org:principis:howdy'... done
Updating metadata for 'rpmfusion-nonfree'... done
Updating metadata for 'updates'... done
Updating metadata for 'enpass'... done
Updating metadata for 'updates-archive'... done
Importing rpm-md... done
rpm-md repo 'rpmfusion-free-updates-testing'; generated: 2024-03-31T11:47:48Z solvables: 6
rpm-md repo 'copr:copr.fedorainfracloud.org:ublue-os:akmods'; generated: 2024-03-17T06:02:48Z solvables: 216
rpm-md repo 'rpmfusion-nonfree-updates'; generated: 2024-03-31T12:14:57Z solvables: 74
rpm-md repo 'cuda-rhel9-x86_64'; generated: 2024-04-03T19:15:05Z solvables: 2134
rpm-md repo 'rpmfusion-free'; generated: 2023-11-04T16:49:08Z solvables: 445
rpm-md repo 'rpmfusion-nonfree-updates-testing'; generated: 2024-03-31T12:15:07Z solvables: 2
rpm-md repo 'cuda-fedora39-x86_64'; generated: 2024-04-03T19:26:21Z solvables: 209
rpm-md repo 'fedora'; generated: 2023-11-01T00:12:39Z solvables: 70825
rpm-md repo 'rpmfusion-free-updates'; generated: 2024-03-31T11:47:41Z solvables: 170
rpm-md repo 'copr:copr.fedorainfracloud.org:principis:howdy'; generated: 2023-08-10T16:15:29Z solvables: 4
rpm-md repo 'rpmfusion-nonfree'; generated: 2023-11-04T17:26:32Z solvables: 208
rpm-md repo 'updates'; generated: 2024-04-04T00:37:30Z solvables: 25477
rpm-md repo 'enpass'; generated: 2023-11-16T10:30:35Z solvables: 33
rpm-md repo 'updates-archive'; generated: 2024-04-04T01:11:00Z solvables: 40351
Resolving dependencies... done
Will download: 75 packages (4.4 GB)
Downloading from 'fedora'... done
Downloading from 'updates'... done
Downloading from 'cuda-fedora39-x86_64'... done
Importing packages... done
Checking out packages... done
error: Checkout cuda-cudart-12-4-12.4.127-1.x86_64: opendir(local): No such file or directory

Reproduction steps

1. Add cuda repo: https://developer.download.nvidia.com/compute/cuda/repos/fedora39/x86_64/cuda-fedora39.repo
2. Run: rpm-ostree install cuda

Expected behavior

Should install cuda without errors.

Actual behavior

Install fails with message:

error: Checkout cuda-cudart-12-4-12.4.127-1.x86_64: opendir(local): No such file or directory

System details

rpm-ostree:
Version: '2024.4'
Git: 900d805f2116135469f31931250a4d14e2be44bb
Features:

• rust
• compose
• container
• fedora-integration

State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: no runs since boot
BootedDeployment:
● ostree-image-signed:docker://ghcr.io/notnotno/custom-kinoite:latest
Digest: sha256:ca8fe64b77708878c28189fecb83b9c507db036d90fe2b978109fa3aa96d8043
Version: 39.20240403.0 (2024-04-04T15:43:06Z)

Additional information

No response

[cgwalters]

Yes, it's because /usr/local is a symlink; we don't make this easily configurable right now.

Also rpm-ostree has some bugs in this space.

Long story short though, if you want to do this:

docker://ghcr.io/notnotno/custom-kinoite:latest

You're already using a custom container image, so you should be able to do this:

RUN rm /usr/local && mkdir /usr/local
RUN rpm-ostree install cuda...

or so.

(And then client side, /usr/local is immutable state note)