Container does not have the cap_net_raw+p capability or setuid? capabilities.

[shabbir-mohammed-sada]

I have deployed the container on to GKE cluster as a sidecar, here is the yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: nginx-netshoot
    labels:
        app: nginx-netshoot
spec:
  replicas: 1
  selector:
    matchLabels:
        app: nginx-netshoot
  template:
      metadata:
       labels:
          app: nginx-netshoot
      spec:
            containers:
            - name: nginx
              image: nginxinc/nginx-unprivileged
              ports:
                  - containerPort: 80
              securityContext:
                runAsNonRoot: true
                runAsUser: 100
                seccompProfile:
                  type: RuntimeDefault
                allowPrivilegeEscalation: false
                capabilities:
                  drop:
                  - NET_RAW
            - name: netshoot
              image: nicolaka/netshoot
              command: ["/bin/bash"]
              args: ["-c", "while true; do ping localhost; sleep 60;done"]
              securityContext:
                runAsNonRoot: true
                runAsUser: 100
                seccompProfile:
                  type: RuntimeDefault
                allowPrivilegeEscalation: false
                capabilities:
                  drop:
                  - NET_RAW
                  - NET_ADMIN

when I check the logs of the container,
I get the following,

ping: socktype: SOCK_RAW
ping: socket: Operation not permitted
ping: => missing cap_net_raw+p capability or setuid?
ping: socktype: SOCK_RAW
ping: socket: Operation not permitted
ping: => missing cap_net_raw+p capability or setuid?

Any ideas on how to overcome this, it looks like it needs packages of libcap/libcap_dev and also tcpdump command gives operation not permitted exception.

[thaJeztah]

Looks like the YAML you posted above drops those capabilities?

capabilities:
  drop:
  - NET_RAW
  - NET_ADMIN
                

[sam1913]

@thaJeztah I have the cis security policies enforced on the cluster is there a workaround to not remove the capabilities and have the tcpdump or ping work?

[thaJeztah]

It may depend on the runtime you're using. Docker 20.10 changed the net.ipv4.ping_group_range sysctl to allow using ping without CAP_NET_RAW (see moby/moby#41030).

I think containerd 1.6 also uses that now (see containerd/containerd#6170), and there's a related discussion in k8s; kubernetes/kubernetes#102612

On a version without that patch (or a runtime that doesn't set this), it will fail with the error you're seeing;

docker run -it --rm --cap-drop CAP_NET_ADMIN --cap-drop CAP_NET_RAW --user=100 nicolaka/netshoot sh -c 'while true; do ping localhost; sleep 60; done'
ping: socktype: SOCK_RAW
ping: socket: Operation not permitted
ping: => missing cap_net_raw+p capability or setuid?

If you change the option (--sysctl net.ipv4.ping_group_range="0 2147483647" on docker run), it works:

docker run -it --rm --cap-drop CAP_NET_ADMIN --cap-drop CAP_NET_RAW --user=100 --sysctl net.ipv4.ping_group_range="0 2147483647" nicolaka/netshoot sh -c 'while true; do ping localhost; sleep 60; done'
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.022 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.076 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.109 ms
64 bytes from localhost (127.0.0.1): icmp_seq=4 ttl=64 time=0.178 ms

(Not exactly sure what the corresponding options are in kubernetes yaml)

[sam1913]

@thaJeztah okay, thank you! I will close this issue now