PSP issues on newly-provisioned AWS EKS Clusters running 1.23+
Closed this issue · 3 comments
New EKS Clusters running version 1.23 and above are configured to use Pod Security Standards and Pod Security Admission by default. This means PodSecurityPolicies are no longer used as they are deprecated as of 1.21 and slated for removal in 1.25.
When you try to install the operator on a new 1.24 k8s cluster you get the following error when the cluster tries to start the registry server:
Message: couldn't ensure registry server - error ensuring pod: : error creating new pod: stocktrader-: pods "stocktrader-7mbvc" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "registry-server" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "registry-server" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "registry-server" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "registry-server" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
Reason: RegistryServerError
At this point I don't know if we need to make a change to the k8s cluster, the operator yaml, or the registry server container to address this issue. We're going do some investigation based on this AWS article on moving to PSS from PSP.
https://aws.amazon.com/blogs/containers/implementing-pod-security-standards-in-amazon-eks/
We ran this command to identify all the PSP items in the cluster:
kubectl get pod -A \
-o jsonpath='{range .items[?(@.metadata.annotations.kubernetes\.io/psp)]}{.metadata.name}{"\t"}{.metadata.annotations.kubernetes\.io/psp}{"\t"}{.metadata.namespace}{"\n"}'aws-node-6zn4s eks.privileged kube-system
aws-node-d2q6f eks.privileged kube-system
aws-node-qw8f4 eks.privileged kube-system
coredns-d5b9bfc4-c7thl eks.privileged kube-system
coredns-d5b9bfc4-lzv9b eks.privileged kube-system
kube-proxy-54sfn eks.privileged kube-system
kube-proxy-d8mq7 eks.privileged kube-system
kube-proxy-gdkmz eks.privileged kube-system
catalog-operator-6f748dc8b9-2547k eks.privileged olm
olm-operator-d79fdcdbb-vmsxx eks.privileged olm
operatorhubio-catalog-7nmtv eks.privileged olm
packageserver-697c5b7597-6s8j5 eks.privileged olm
packageserver-697c5b7597-snwnq eks.privileged olm
Fixed in v1.0.0 of the operator