OpenShift 4 - builds don't work if whitelist in registrySources defined.

[vadimzharov]

I have OCP4.1 RC installed:
$ oc4 get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.1.0-rc.0 True False 8d Cluster version is 4.1.0-rc.0

And have image policies implemented with list of allowed registries as build source:

apiVersion: config.openshift.io/v1
kind: Image
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"config.openshift.io/v1","kind":"Image","metadata":{"annotations":{},"name":"cluster","namespace":""},"spec":{"allowedRegistriesForImport":[{"domainName":"registry.redhat.io","insecure":false},{"domainName":"quay.io","insecure":false}],"registrySources":{"allowedRegistries":["registry.redhat.io","registry.access.redhat.com","quay.io"]}}}
    release.openshift.io/create-only: "true"
  name: cluster
spec:
  allowedRegistriesForImport:
  - domainName: registry.redhat.io
    insecure: false
  - domainName: quay.io
    insecure: false
  - domainName: registry.access.redhat.com
    insecure: false
  registrySources:
    allowedRegistries:
    - registry.redhat.io
    - registry.access.redhat.com
    - image-registry.openshift-image-registry.svc:5000
    - quay.io
status:
  internalRegistryHostname: image-registry.openshift-image-registry.svc:5000

All my builds (for example sample Ruby application build) are failing with same error:

error: build error: error copying layers and metadata for container "ceed796969e4947a471e4c866606d1fb5067055f7c0d8d9e3b174e3906fa37d7": Source image rejected: Running image containers-storage:ruby-working-container is rejected by policy.

Builder pod has this generated configuration:

{"default":[{"type":"reject"}],"transports":{"atomic":{"image-registry.openshift-image-registry.svc:5000":[{"type":"insecureAcceptAnything"}],"quay.io":[{"type":"insecureAcceptAnything"}],"registry.access.redhat.com":[{"type":"insecureAcceptAnything"}],"registry.redhat.io":[{"type":"insecureAcceptAnything"}]},"docker":{"image-registry.openshift-image-registry.svc:5000":[{"type":"insecureAcceptAnything"}],"quay.io":[{"type":"insecureAcceptAnything"}],"registry.access.redhat.com":[{"type":"insecureAcceptAnything"}],"registry.redhat.io":[{"type":"insecureAcceptAnything"}]}}}

with default type - reject.
As I understand, if default=reject it's also applies to all types of transport, including "containers-storage" - and this can be the reason why my builds failing.

If I tweak policy.json file (in builder pod, using oc debug) by adding containers-storage into transport:

sh-4.2# cat policy.json 
{
        "default": [{
                "type": "reject"
        }],
        "transports": {
                "atomic": {
                        "image-registry.openshift-image-registry.svc:5000": [{
                                "type": "insecureAcceptAnything"
                        }],
                        "quay.io": [{
                                "type": "insecureAcceptAnything"
                        }],
                        "registry.access.redhat.com": [{
                                "type": "insecureAcceptAnything"
                        }],
                        "registry.redhat.io": [{
                                "type": "insecureAcceptAnything"
                        }]
                },
                "docker": {
                        "image-registry.openshift-image-registry.svc:5000": [{
                                "type": "insecureAcceptAnything"
                        }],
                        "quay.io": [{
                                "type": "insecureAcceptAnything"
                        }],
                        "registry.access.redhat.com": [{
                                "type": "insecureAcceptAnything"
                        }],
                        "registry.redhat.io": [{
                                "type": "insecureAcceptAnything"
                        }]
                },
                "containers-storage": {
                        "": [{
                                "type": "insecureAcceptAnything"
                        }]
                }
        }
}

Then build finished successfully.

With blacklist in image policy configuration:

kind: Image
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"config.openshift.io/v1","kind":"Image","metadata":{"annotations":{},"name":"cluster","namespace":""},"spec":{"allowedRegistriesForImport":[{"domainName":"registry.redhat.io","insecure":false},{"domainName":"quay.io","insecure":false}],"registrySources":{"allowedRegistries":["registry.redhat.io","registry.access.redhat.com","quay.io"]}}}
    release.openshift.io/create-only: "true"
  name: cluster
spec:
  allowedRegistriesForImport:
  - domainName: registry.redhat.io
    insecure: false
  - domainName: quay.io
    insecure: false
  - domainName: registry.access.redhat.com
    insecure: false
  registrySources:
    blockedRegistries:
    - docker.io

All builds complete successfuly. With this settings policy.json in builder pod is:

{"default":[{"type":"insecureAcceptAnything"}],"transports":{"atomic":{"docker.io":[{"type":"reject"}]},"docker":{"docker.io":[{"type":"reject"}]}}}

with default type insecureAcceptAnything - which allows container tools to use transport "containers-storage".

[adambkaplan]

@nalind I'm pretty sure this is a bug - seems that builds will break if we don't allow containers-storage.