Non existing localhostProfile Seccomp profile is not applied on Kubernetes nodes >= 1.28

[MetalPinguinInc]

What happened?

I am running 2 clusters, 1 is still on 1.26 and the other is on 1.29.5. I tried to apply a custom seccomp profile to a pod on 1.29.5 and noticed it did not seem to work. While trying to pin-point the issue, I found out that applying a custom seccomp profile that does not exist (i.e. the file does not exist on the node) to a pod on a kubernetes node below 1.28 fails to start the pod, with an error:

Error: failed to generate security options for container "test-container": failed to generate seccomp security options for container: cannot load seccomp profile "/var/lib/kubelet/seccomp/profiles/audit.json": open /var/lib/kubelet/seccomp/profiles/audit.json: no such file or directory

However, when trying to do the same on a kubernetes node >= 1.28 incorrectly starts the pod, with no mention of the seccomp profile file not being found or being invalid. Although I used minikube to reproduce this bug, I observed it first on a bare metal installation.

What did you expect to happen?

I expect a pod not to start when the seccomp profile cannot be loaded.

How can we reproduce it (as minimally and precisely as possible)?

minikube start --kubernetes-version 1.27
kubectl apply -f audit-pod.yaml

Observe that the pod does not start and gives an error:

Error: failed to generate security options for container "test-container": failed to generate seccomp security options for container: cannot load seccomp profile "/var/lib/kubelet/seccomp/profiles/audit.json": open /var/lib/kubelet/seccomp/profiles/audit.json: no such file or directory

minikube delete
minikube start --kubernetes-version 1.28\
kubectl apply -f audit-pod.yaml\

Observe that the pod has started without any errors.

audit-pod.yaml:

kind: Pod
metadata:
  name: audit-pod
  labels:
    app: audit-pod
spec:
  securityContext:
    seccompProfile:
      type: Localhost
      localhostProfile: profiles/audit.json
  containers:
  - name: test-container
    image: hashicorp/http-echo:1.0
    args:
    - "-text=just made some syscalls!"
    securityContext:
      allowPrivilegeEscalation: false

Anything else we need to know?

No response

Kubernetes version

Version that errors on invalid seccomp profile:

Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.7
WARNING: version difference between client (1.29) and server (1.27) exceeds the supported minor version skew of +/-1

Version that does not error:

$ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.3

Cloud provider

None

OS version

# On Linux:
$ cat /etc/os-release
NAME=Buildroot
VERSION=2021.02.12-1-gb75713b-dirty
ID=buildroot
VERSION_ID=2021.02.12
PRETTY_NAME="Buildroot 2021.02.12"
$ uname -a
Linux minikube 5.10.57 #1 SMP Tue Nov 7 06:51:54 UTC 2023 x86_64 GNU/Linux

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[neolit123]

/sig node security