Several test scenarios were executed against a Kubernetes 1.23 cluster, with Pod Security Admission (PSA) and Pod Security Standards (PSS) Privileged profile enabled by default. The testing was designed to exercise different PSA modes and PSS profiles, while producing the following responses:
- Allowing Pods that meet profile requirements
- Disallowing pods that don’t meet profile requirements
- Allowing Deployments, even if Pods did not meet PSS profile requirements
- Failure (forbidden) responses to Kubernetes API server clients when Pods don’t meet profile requirements
- Deployment resource statuses, reflecting failed (forbidden) Pods
- Kubernetes API server logs with PSA controller responses, reflecting failed (forbidden) Pods
- Warning messages to Kubernetes API server clients when Deployments contained Pod specs that failed PSS profile requirements
- PSA functions correctly in Kubernetes 1.23
- PSS profiles (Privileged, Baseline, and Restricted) function as expected in Kubenetes 1.23
- PSA modes (audit, enforce, and warn) function as expected in Kubernetes 1.23
- PSA Enforce mode only affects Pods, and does not affect workload resource controllers (Deployment, etc.) that create Pods.
- No PSA exemptions are configured at API server startup, for the PSA controller.
- The Privileged PSS profile is configured by default for all PSA modes, and set to latest versions.
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1beta1
kind: PodSecurityConfiguration
# Defaults applied when a mode label is not set.
#
# Level label values must be one of:
# - "privileged" (default)
# - "baseline"
# - "restricted"
#
# Version label values must be one of:
# - "latest" (default)
# - specific version like "v1.24"
defaults:
enforce: "privileged"
enforce-version: "latest"
audit: "privileged"
audit-version: "latest"
warn: "privileged"
warn-version: "latest"
exemptions:
# Array of authenticated usernames to exempt.
usernames: []
# Array of runtime class names to exempt.
runtimeClasses: []
# Array of namespaces to exempt.
namespaces: []
- policy-test Namespace created
apiVersion: v1
kind: Namespace
metadata:
name: policy-test
labels:
# pod-security.kubernetes.io/enforce: privileged
# pod-security.kubernetes.io/audit: privileged
# pod-security.kubernetes.io/warn: privileged
# pod-security.kubernetes.io/enforce: baseline
# pod-security.kubernetes.io/audit: baseline
# pod-security.kubernetes.io/warn: baseline
# pod-security.kubernetes.io/enforce: restricted
# pod-security.kubernetes.io/audit: restricted
# pod-security.kubernetes.io/warn: restricted
- Known good Kubernetes Deployment created and then deleted
apiVersion: apps/v1
kind: Deployment
namespace: policy-test
...
spec:
containers:
- name: test
image: public.ecr.aws/r2l1x4g2/go-http-server:v0.1.0-23ffe0a715
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
runAsUser: 1000
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop: ["ALL"]
seccompProfile:
type: "RuntimeDefault"
ports:
- containerPort: 8080
...
- Known bad Kubernetes Deployment created
- No securityContext element at the Pod level
- No securityContext element at the Container level
apiVersion: apps/v1
kind: Deployment
metadata:
name: test
namespace: policy-test
...
spec:
containers:
- name: test
image: public.ecr.aws/r2l1x4g2/go-http-server:v0.1.0-23ffe0a715
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
...
- Known bad Pod created
- No securityContext element at the Pod level
- No securityContext element at the Container level
apiVersion: v1
kind: Pod
metadata:
name: test
namespace: policy-test
...
spec:
containers:
- name: test
image: public.ecr.aws/r2l1x4g2/go-http-server:v0.1.0-23ffe0a715
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
...
- Known bad Pod created
- securityContext element at the Pod level exists with valid runAsUser and runAsNonRoot elements.
- No securityContext element at the Container level
apiVersion: v1
kind: Pod
metadata:
name: test2
namespace: policy-test
...
spec:
securityContext:
runAsUser: 1000
runAsNonRoot: true
containers:
- name: test
image: public.ecr.aws/r2l1x4g2/go-http-server:v0.1.0-23ffe0a715
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
...
- Known bad Pod created
- No securityContext element at the Pod level.
- securityContext element at the Container level exists with incorrect settings for allowPrivilegeEscalation, readOnlyRootFilesystem, and runAsNonRoot
apiVersion: v1
kind: Pod
metadata:
name: test3
namespace: policy-test
...
spec:
containers:
- name: test
image: public.ecr.aws/r2l1x4g2/go-http-server:v0.1.0-23ffe0a715
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: true
runAsUser: 1000
readOnlyRootFilesystem: false
runAsNonRoot: false
capabilities:
drop: ["ALL"]
seccompProfile:
type: "RuntimeDefault"
ports:
...
- Known bad Pod created
- No securityContext element at the Pod level.
- securityContext element at the Container level exists with correct settings
- Pod spec has incorrect hostNetwork, hostPID, and hostIPC settings
apiVersion: v1
kind: Pod
metadata:
name: test4
namespace: policy-test
...
spec:
hostNetwork: true
hostPID: true
hostIPC: true
containers:
- name: test
image: public.ecr.aws/r2l1x4g2/go-http-server:v0.1.0-23ffe0a715
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
runAsUser: 1000
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop: ["ALL"]
seccompProfile:
type: "RuntimeDefault"
ports:
- containerPort: 8080
...
- Namespace Config (no Namespace-level settings)
apiVersion: v1
kind: Namespace
metadata:
name: policy-test
labels:
# pod-security.kubernetes.io/enforce: privileged
# pod-security.kubernetes.io/audit: privileged
# pod-security.kubernetes.io/warn: privileged
# pod-security.kubernetes.io/enforce: baseline
# pod-security.kubernetes.io/audit: baseline
# pod-security.kubernetes.io/warn: baseline
# pod-security.kubernetes.io/enforce: restricted
# pod-security.kubernetes.io/audit: restricted
# pod-security.kubernetes.io/warn: restricted
- Test Output - Default PSS Privileged Profile Applied (5 Pods allowed, 0 Pods disallowed)
namespace/policy-test created
>>> 1. Good config...
deployment.apps/test created
deployment.apps "test" deleted
>>> 2. Deployment - Missing container security context element...
deployment.apps/test created
>>> 3. Pod - Missing container security context element...
pod/test created
>>> 4. Pod - Pod security context, but Missing container security context element...
pod/test2 created
>>> 5. Pod - Container security context element present, with incorrect settings...
pod/test3 created
>>> 6. Pod - Container security context element present, with incorrect spec.hostNetwork, spec.hostPID, spec.hostIPC settings...
pod/test4 created
k -n policy-test get po
NAME READY STATUS RESTARTS AGE
test 1/1 Running 0 70s
test-59955f994-djtqz 1/1 Running 0 76s
test2 1/1 Running 0 65s
test3 1/1 Running 0 59s
test4 1/1 Running 0 55s
- Namespace Config
apiVersion: v1
kind: Namespace
metadata:
name: policy-test
labels:
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/audit: privileged
pod-security.kubernetes.io/warn: privileged
# pod-security.kubernetes.io/enforce: baseline
# pod-security.kubernetes.io/audit: baseline
# pod-security.kubernetes.io/warn: baseline
# pod-security.kubernetes.io/enforce: restricted
# pod-security.kubernetes.io/audit: restricted
# pod-security.kubernetes.io/warn: restricted
- Test Output - Namespace-level PSS Privileged Profile Applied (5 Pods allowed, 0 Pods disallowed)
namespace/policy-test created
>>> 1. Good config...
deployment.apps/test created
deployment.apps "test" deleted
>>> 2. Deployment - Missing container security context element...
deployment.apps/test created
>>> 3. Pod - Missing container security context element...
pod/test created
>>> 4. Pod - Pod security context, but Missing container security context element...
pod/test2 created
>>> 5. Pod - Container security context element present, with incorrect settings...
pod/test3 created
>>> 6. Pod - Container security context element present, with incorrect spec.hostNetwork, spec.hostPID, spec.hostIPC settings...
pod/test4 created
k -n policy-test get po
NAME READY STATUS RESTARTS AGE
test 1/1 Running 0 54s
test-59955f994-cssz9 1/1 Running 0 59s
test2 1/1 Running 0 48s
test3 1/1 Running 0 42s
test4 1/1 Running 0 38s
- Namespace Config
apiVersion: v1
kind: Namespace
metadata:
name: policy-test
labels:
# pod-security.kubernetes.io/enforce: privileged
# pod-security.kubernetes.io/audit: privileged
# pod-security.kubernetes.io/warn: privileged
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/audit: baseline
pod-security.kubernetes.io/warn: baseline
# pod-security.kubernetes.io/enforce: restricted
# pod-security.kubernetes.io/audit: restricted
# pod-security.kubernetes.io/warn: restricted
- Test Output - Namespace-level PSS Baseline Profile Applied (4 Pods allowed, 1 Pod disallowed)
namespace/policy-test created
>>> 1. Good config...
deployment.apps/test created
deployment.apps "test" deleted
>>> 2. Deployment - Missing container security context element...
deployment.apps/test created
>>> 3. Pod - Missing container security context element...
pod/test created
>>> 4. Pod - Pod security context, but Missing container security context element...
pod/test2 created
>>> 5. Pod - Container security context element present, with incorrect settings...
pod/test3 created
>>> 6. Pod - Container security context element present, with incorrect spec.hostNetwork, spec.hostPID, spec.hostIPC settings...
Error from server (Forbidden): error when creating "policy/psa-pss/tests/6-pod.yaml": pods "test4" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true, hostPID=true, hostIPC=true), hostPort (container "test" uses hostPort 8080)
k -n policy-test get po
NAME READY STATUS RESTARTS AGE
test 1/1 Running 0 46s
test-59955f994-6tbj7 1/1 Running 0 52s
test2 1/1 Running 0 42s
test3 1/1 Running 0 37s
- Namespace Config
apiVersion: v1
kind: Namespace
metadata:
name: policy-test
labels:
# pod-security.kubernetes.io/enforce: privileged
# pod-security.kubernetes.io/audit: privileged
# pod-security.kubernetes.io/warn: privileged
# pod-security.kubernetes.io/enforce: baseline
# pod-security.kubernetes.io/audit: baseline
# pod-security.kubernetes.io/warn: baseline
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/warn: restricted
- Test Output - Namespace-level PSS Restricted Profile Applied (0 Pods allowed, 5 Pods disallowed)
- 1 Deployment created, with 0 Pods allowed
namespace/policy-test created
>>> 1. Good config...
deployment.apps/test created
deployment.apps "test" deleted
>>> 2. Deployment - Missing container security context element...
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "test" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "test" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "test" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "test" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
deployment.apps/test created
>>> 3. Pod - Missing container security context element...
Error from server (Forbidden): error when creating "policy/psa-pss/tests/3-pod.yaml": pods "test" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "test" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "test" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "test" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "test" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
>>> 4. Pod - Pod security context, but Missing container security context element...
Error from server (Forbidden): error when creating "policy/psa-pss/tests/4-pod.yaml": pods "test2" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "test" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "test" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "test" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
>>> 5. Pod - Container security context element present, with incorrect settings...
Error from server (Forbidden): error when creating "policy/psa-pss/tests/5-pod.yaml": pods "test3" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "test" must set securityContext.allowPrivilegeEscalation=false), runAsNonRoot != true (container "test" must not set securityContext.runAsNonRoot=false)
>>> 6. Pod - Container security context element present, with incorrect spec.hostNetwork, spec.hostPID, spec.hostIPC settings...
Error from server (Forbidden): error when creating "policy/psa-pss/tests/6-pod.yaml": pods "test4" is forbidden: violates PodSecurity "restricted:latest": host namespaces (hostNetwork=true, hostPID=true, hostIPC=true), hostPort (container "test" uses hostPort 8080)
k -n policy-test get po
No resources found in policy-test namespace.
k -n policy-test get deploy test -oyaml
...
status:
conditions:
...
- lastTransitionTime: "2022-07-12T23:56:10Z"
lastUpdateTime: "2022-07-12T23:56:10Z"
message: 'pods "test-59955f994-wl8hf" is forbidden: violates PodSecurity "restricted:latest":
allowPrivilegeEscalation != false (container "test" must set securityContext.allowPrivilegeEscalation=false),
unrestricted capabilities (container "test" must set securityContext.capabilities.drop=["ALL"]),
runAsNonRoot != true (pod or container "test" must set securityContext.runAsNonRoot=true),
seccompProfile (pod or container "test" must set securityContext.seccompProfile.type
to "RuntimeDefault" or "Localhost")'
reason: FailedCreate
status: "True"
type: ReplicaFailure
...
kubectl label --overwrite ns policy-test pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/warn=restricted pod-security.kubernetes.io/audit=restricted
kubectl get ns policy-test -L pod-security.kubernetes.io/enforce \
-L pod-security.kubernetes.io/warn -L pod-security.kubernetes.io/audit
- https://aws.github.io/aws-eks-best-practices/security/docs/pods/#pod-security-standards-pss-and-pod-security-admission-psa
- https://kubernetes.io/docs/concepts/security/pod-security-standards/
- https://kubernetes.io/docs/concepts/security/pod-security-admission/
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.