containers/storage

additionalimagestores Not Working as Expected

kasualkeef opened this issue · 8 comments

kasualkeef commented

Issue Description

I have been able to successfully follow the guide for setting up a read-only image store as described here:
https://www.redhat.com/sysadmin/image-stores-podman

Unfortunately, the store does not seem read-only at all. If the user does not have write permissions to the files on the image store, he is unable to build images using a store image as the FROM base.

We are running rootless Podman on RHEL 8 with all of the latest patches from the RHEL repos.

Steps to reproduce the issue

Steps to reproduce the issue

  1. Pull down images with the --root flag as described here.

    • This must be done as a rootless user or the in-container permissions will be completely broken

  2. chmod the files so that other users can read the files

    • We have the force_mask='shared' storage.conf setting but this still doesn't seem to affect the file mode on the pulled files (they're still 0700)

  3. At this stage, another rootless user on the system can point additionalimagestores to the image store directory and do simple things like view the images and run a container instance. However, image builds do not work correctly unless the rootless user has full write permissions to the image store.

     ## Pull Image into RO Store
 [rootless.user@rhel8dev01 ~]$ podman pull --root /data/image-cache/ harbor.domain.com/dod-ironbank/ironbank/redhat/ubi/ubi8
 Trying to pull harbor.domain.com/dod-ironbank/ironbank/redhat/ubi/ubi8:latest...
 Getting image source signatures
 Copying blob 1ea03711adea done  
 Copying blob e5dd65eaa632 done  
 Copying config b7c4db37ab done  
 Writing manifest to image destination
 b7c4db37ab5f29a75921f41120161a998149ac52f4605b3acd46325c08f903ba
 [rootless.user@rhel8dev01 ~]$ ls -l /data/image-cache/
 total 8
 drwx------. 2 rootless.user admins   27 Feb 12 07:24 libpod
 drwx------. 5 rootless.user admins  185 Feb 12 07:24 overlay
 drwx------. 2 rootless.user admins   29 Feb 12 07:24 overlay-containers
 drwx------. 3 rootless.user admins  116 Feb 12 07:24 overlay-images
 drwx------. 2 rootless.user admins 4096 Feb 12 07:24 overlay-layers
 -rw-r--r--. 1 rootless.user admins   64 Feb 12 07:24 storage.lock
 -rw-r--r--. 1 rootless.user admins    0 Feb 12 07:24 userns.lock
 [rootless.user@rhel8dev01 ~]$ chmod -R a+rX /data/image-cache/
 
 ## View RO Images
 [rootless.user@rhel8dev01 ~]$ su - other.rootless.user
 [other.rootless.user@rhel8dev01 test]$ podman images
 REPOSITORY                                                TAG         IMAGE ID      CREATED     SIZE        R/O
 harbor.example.com/dod-ironbank/ironbank/redhat/ubi/ubi8  latest      b7c4db37ab5f  2 days ago  246 MB      true
 
 ## Test Image Build
 [other.rootless.user@rhel8dev01 test]$ cat Dockerfile 
 FROM harbor.example.com/dod-ironbank/ironbank/redhat/ubi/ubi8
 
 RUN echo foo
 [other.rootless.user@rhel8dev01 test]$ podman build -f Dockerfile . -t test_build:latest
 STEP 1/2: FROM harbor.example.com/dod-ironbank/ironbank/redhat/ubi/ubi8
 STEP 2/2: RUN echo foo
 foo
 COMMIT test_build:latest
 Error: committing container for step {Env:[container=oci PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[echo foo] Flags:[] Attrs:map[] Message:RUN echo foo Original:RUN echo foo}: copying layers and metadata for container "3c689cd9949639fa75e3da707a574fe529c3004a56c9476d4fb2dc295f9ec353": initializing source containers-storage:ubi8-working-container: extracting layer "f8f29dbe2320223cd8c3e725967c24b24f1cce2bbb8c88ba42e715ecf76b0de6": chown /data/image-cache/overlay/ed9962cc48ff67290ea405d51a5cfa81069243fae21ff51857a754a17c2fbbaf/diff: operation not permitted
 
 ## Give User Write Perms and Try Build Again
 [root@rhel8dev01 ~]# chown -R other.rootless.user /data/image-cache/
 [root@rhel8dev01 ~]# su - other.rootless.user
 [other.rootless.user@rhel8dev01 ~]$ cd dev/test/
 [other.rootless.user@rhel8dev01 test]$ podman build -f Dockerfile . -t test_build:latest
 STEP 1/2: FROM harbor.example.com/dod-ironbank/ironbank/redhat/ubi/ubi8
 STEP 2/2: RUN echo foo
 foo
 COMMIT test_build:latest
 --> d71e8c5a6cee
 Successfully tagged localhost/test_build:latest
 d71e8c5a6ceea8858b99d3612f81ac2b603598d12f8c2c80cf43bd9423b8ccc2
 [other.rootless.user@rhel8dev01 ~]$

Describe the results you received

I see permissions errors on container image builds based on the RO image cache.

Describe the results you expected

I expect the additionalimagestores images to be read-only and useful for container image builds.

podman info output

[rootless.user@rhel8dev01 test]$ podman info
host:
  arch: amd64
  buildahVersion: 1.31.3
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.8-1.module+el8.9.0+21243+a586538b.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: 64375b77dc0eda8f5a71b9208e041bc1e36eac81'
  cpuUtilization:
    idlePercent: 99.14
    systemPercent: 0.1
    userPercent: 0.76
  cpus: 12
  databaseBackend: boltdb
  distribution:
    distribution: '"rhel"'
    version: "8.9"
  eventLogger: file
  freeLocks: 2048
  hostname: rhel8dev01
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1941000000
      size: 1
    - container_id: 1
      host_id: 2918049
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1941000003
      size: 1
    - container_id: 1
      host_id: 2918049
      size: 65536
  kernel: 4.18.0-513.11.1.el8_9.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 6334828544
  memTotal: 10438213632
  networkBackend: cni
  networkBackendInfo:
    backend: cni
    dns:
      package: podman-plugins-4.6.1-8.module+el8.9.0+21243+a586538b.x86_64
      path: /usr/libexec/cni/dnsname
      version: |-
        CNI dnsname plugin
        version: 1.3.1
        commit: unknown
    package: containernetworking-plugins-1.3.0-8.module+el8.9.0+21243+a586538b.x86_64
    path: /usr/libexec/cni
  ociRuntime:
    name: runc
    package: runc-1.1.12-1.module+el8.9.0+21243+a586538b.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.1.12
      spec: 1.0.2-dev
      go: go1.20.12
      libseccomp: 2.5.2
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    path: /run/user/1941000003/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_SYS_CHROOT,CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.1-1.module+el8.9.0+21243+a586538b.x86_64
    version: |-
      slirp4netns version 1.2.1
      commit: 09e31e92fa3d2a1d3ca261adaeb012c8d75a8194
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 6462369792
  swapTotal: 6462369792
  uptime: 4h 11m 6.00s (Approximately 0.17 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /home/users/rootless.user/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /data/image-cache
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.12-1.module+el8.9.0+21243+a586538b.x86_64
      Version: |-
        fusermount3 version: 3.3.0
        fuse-overlayfs: version 1.12
        FUSE library version 3.3.0
        using FUSE kernel interface version 7.26
  graphRoot: /data/podman_users/rootless.user/share/containers/storage
  graphRootAllocated: 268287614976
  graphRootUsed: 2173411328
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /data/scratch/
  imageStore:
    number: 2
  runRoot: /data/podman_users/rootless.user/containers
  transientStore: false
  volumePath: /data/podman_users/rootless.user/share/containers/storage/volumes
version:
  APIVersion: 4.6.1
  Built: 1707224641
  BuiltTime: Tue Feb  6 08:04:01 2024
  GitCommit: ""
  GoVersion: go1.20.12
  Os: linux
  OsArch: linux/amd64
  Version: 4.6.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

No

Additional environment details

The storage.conf configuration is as follows:

[storage]
  driver = "overlay"
  runroot = "/data/podman_users/$USER/containers"
  graphroot = "/data/podman_users/$USER/share/containers/storage"
  rootless_storage_path = "/data/podman_users/$USER/share/containers/storage"
  [storage.options]
    additionalimagestores = ['/data/image-cache']   
    size = ""
    remap-uids = ""
    remap-gids = ""
    ignore_chown_errors = ""
    remap-user = ""
    remap-group = ""
    skip_mount_home = ""
    mount_program = "/usr/bin/fuse-overlayfs"
    mountopt = ""
    [storage.options.overlay]
      ignore_chown_errors = ""
      mountopt = ""
      mount_program = "/usr/bin/fuse-overlayfs"
      force_mask = "shared"
      size = ""
      skip_mount_home = ""

Additional information

No response

giuseppe commented

opened a PR: #1828