/container-security-checklist

Checklist for container security - devsecops practices

Apache License 2.0Apache-2.0

Container Security Checklist: From the image to the workload

Table Of Contents

Cloud Native challenges

Legacy apps Cloud Native apps Cloud Native Security
Discrete, infrequent releases frequent releases, using CI/CD Shifting left with automated testing
Very little open source Open source everywhere SCA - Software composition analysis
Proprietary software Proprietary code, Open source, Third-party software Software supply chain risk
Persistent workloads Ephemeral workloads. Ensure that your containers are stateless and immutable Runtime controls that follow the workload
Hypervisor or hardware isolation Shared kernel, obscured OS Enforce least privilege on each workload
Permanent address Orchestrated containers. Kubernetes creates DNS records for services and pods Identity-based segmentation
Vertical control of the stack multi-cloud Detecting cloud services misconfigurations (CSPM)
Networking monitoring and threat detection tools were based on auditd, syslog, dead-disk forensics, and it used to get the full contents of network packets to disk "packet captures". Capturing packets sotes every packet in a network to disk and runs custom pattern matching on each packet to identify an attack. Cloud native apps the traffic is encrypted. Packet captures are too costly and ineffective for cloud native environments. Using eBPF programs, you collect the events in real time without disruption to the app.

Table by Aqua Cloud Native Security Platform, more details download here

Container Threat Model

thread-model Figure by Container Security by Liz Rice

  • Insecure Host
  • Misconfiguration container
  • Vulnerable application
  • Supply chain attacks
  • Expose secrets
  • Insecure networking
  • Integrity and confidentiality of OS images
  • Container escape vulnerabilities

Container Security Checklist

Checklist to build and secure the images across the following phases:

Build Figure by cncf/tag-security

Supply Chain Security

  • Enforce image trust with Image signing: Container Signing, Verification and Storage in an OCI registry.

Image Signing

Sign and verify images to mitigate a man in the middle (MITM) attack. Docker offers a Content Trust mechanism that allows you to cryptographically sign images using a private key. This guarantees the image, and its tags, have not been modified.

Secure the Build

Hardening Code - Secure SDLC (Software Development Life Cycle)

  • Do a static analysis of the code and libraries used by the code to surface any vulnerabilities in the code and its dependencies.

Secure the Image - Hardening

You can build the container images using Docker, Kaniko.

  • Reduce the attack surface

Package a single application per container. Small container images. Minimize the number of layers.

  • Multi-staged builds.

A well-designed multi-stage build contains only the minimal binary files and dependencies required for the final image, with no build tools or intermediate files. Optimize cache.

  • Use official base images.
    • Avoid unknown public images.
  • Rootless. Run as a non-root user. Least privileged user
  • Create a dedicated user and group on the image.

Do not use a UID below 10,000. For best security, always run your processes as a UID above 10,000. Remove setuid and setgid permissions from the images

  • Avoid privileged containers, which lets a container run as root on the local machine.
  • Use only the necessary Privileged Capabilities.
    • Drop kernel modules, system time, trace processes (CAP_SYS_MODULE, CAP_SYS_TIME, CAP_SYS_PTRACE ).
  • Enable the --read-only mode in docker, if it's possible.
  • Don't leave sensitive information (secrets, tokens, keys, etc) in the image.
  • Not mounting Host Path.
  • Use Metadata Labels for Images, such as licensing information, sources, names of authors, and relation of containers to projects or components.
  • Used fixed image tag for inmutability. 
Pulling images by digest
docker images --digests
docker pull alpine@sha256:b7233dafbed64e3738630b69382a8b231726aa1014ccaabc1947c5308a8910a7

  • Enabled Security profiles: SELinux, AppArmor, Seccomp.

  • Static code analysis tool for Dockerfile like a linter. Detect misconfigurations

    • Hadolint
    • Packers (including encrypters), and downloaders are all able to evade static scanning by, for example, encrypting binary code that is only executed in memory, making the malware active only in runtime.
    • Trivy detect missconfigurations

Image Scanning

  • Scan image for Common Vulnerabilities and Exposures (CVE)
  • Check image for secrets
  • Prevent attacks using the Supply Chain Attack
  • Used dynamic analysis techniques for containers to prevent runtime bad behaviours.

Container Security Scanners

Comparing the container scanners results:

More Material about build containers

Secure the Container Registry

Best configurations with ECR, ACR, Harbor, etc. Best practices.

  • Lock down access to the image registry (who can push/pull) to restrict which users can upload and download images from it. Uses Role Based Access Control (RBAC)

There is no guarantee that the image you are pulling from the registry is trusted. It may unintentionally contain security vulnerabilities, or may have intentionally been replaced with an image compromised by attackers.

  • Use a private registry deployed behind firewall, to reduce the risk of tampering.

Registry Resources

Secure the Container Runtime

See the following container runtimes, there are three main types of container runtimes—low-level runtimes, high-level runtimes, and virtualized runtimes or sandboxed.

  1. Low-Level Container Runtimes:
  1. High-Level Container Runtimes
  1. Sandboxed and Virtualized Container Runtimes

Why is important Runtime Security?

  • Detection of IOC (Indicator Of Compromise)
  • Detect Zero Days attack
  • Compliance requirement
  • Recommended in highly dynamic environments

Constraints

  • Event context
  • Safety
  • Low overhead
  • Wide support of kernels

Enable detection of anomalous behaviour in applications.

  • Applied the secure configurations in the container runtime. By default is insecure.
  • Restrict access to container runtime daemon/APIs
  • Use if it's possible in Rootless Mode.

Docker Security

  • Avoid misconfigured exposed Docker API Ports, attackers used the misconfigured port to deploy and run a malicious image that contained malware that was specifically designed to evade static scanning.
  • TLS encryption between the Docker client and daemon. Do not expose the Docker engine using Unix socket or remotely using http.

Never make the daemon socket available for remote connections, unless you are using Docker’s encrypted HTTPS socket, which supports authentication.

  • Limit the usage of mount Docker socket in a container in an untrusted environment.

  • Do not run Docker images with an option that exposes the socket in the container.

    -v /var/run/docker.sock://var/run/docker.sock

The Docker daemon socket is a Unix network socket that facilitates communication with the Docker API. By default, this socket is owned by the root user. If anyone else obtains access to the socket, they will have permissions equivalent to root access to the host.

  • Run Docker in Rootless Mode. docker context use rootless

  • Enable the user namespaces.

  • Enable Docker Content Trust. Docker. DOCKER_CONTENT_TRUST=1 . Docker Content Trust implements The Update Framework (TUF) . Powered by Notary, an open-source TUF-client and server that can operate over arbitrary trusted collections of data.

  • Do not run Docker without the default seccomp profile: seccomp=unconfined

More Material about Docker Security

Secure the Infrastructure

Risk:

  • If host is compromised, the container will be too.
  • Kernel exploits

Best practices:

  • Keep the host kernel patched to prevent a range of known vulnerabilities, many of which can result in container escape. Since the kernel is shared by the container and the host, the kernel exploits when an attacker manages to run on a container can directly affect the host.

  • Use CIS-Benchmark for the operating system.

  • Use secure computing (seccomp) to restrict host system call (syscall) access from containers.

  • Use Security-Enhanced Linux (SELinux) to further isolate containers.

Secure the Data

  • Don't leak sensitive info in the images, avoid using environment variables for your sensitive information.

Secrets are Digital credentials:

  • passwords
  • API keys & Tokens
  • SSH keys
  • Private certificates for secure communication, transmitting and receiving data (TLS, SSL, and so on)
  • Private encryption keys for systems like PGP
  • Database names or connection strings.
  • Sensitive configuration settings (email address, usernames, debug flags, etc.)

  • Use a proper filesystem encryption technology for container storage

  • Use volume mounts to pass secrets to a container at runtime

  • Provide write/execute access only to the containers that need to modify the data in a specific host filesystem path

  • OPA to write controls like only allowing Read-only Root Filesystem access, listing allowed host filesystem paths to mount, and listing allowed Flex volume drivers.

  • Automatically scan container images for sensitive data such as credentials, tokens, SSH keys, TLS certificates, database names or connection strings and so on, before pushing them to a container registry (can be done locally and in CI).

  • Limit storage related syscalls and capabilities to prevent runtime privilege escalation.

  • Implement RBAC, or role-based access control. Every human or application only needs the minimum secrets required to operate, nothing more. Principle of Least Privilege.

  • Run audits regularly. Centralized audit trails are the key to knowing all the key security events.

  • Rotate secrets, a standard security practice.

  • Automatically create and store secrets

Secrets Management Tools

Open source tools:

Cloud Provider Key Management

Enterprise secrets vault:

Secure the Workloads... Running the containers

  • Avoid privileged containers

    • Root access to all devices • Ability to tamper with Linux security modules like AppArmor and SELinux • Ability to install a new instance of the Docker platform, using the host’s kernel capabilities, and run Docker within Docker.

To check if the container is running in privileged mode docker inspect --format =’{{. HostConfig.Privileged}}’[container_id]

  • Limit container resources.

When a container is compromised, attackers may try to make use of the underlying host resources to perform malicious activity. Set memory and CPU usage limits to minimize the impact of breaches for resource-intensive containers.

docker run -d --name container-1 --cpuset-cpus 0 --cpu-shares 768 cpu-stress

  • Preventing a fork bomb. docker run --rm -it --pids-limit 200 debian:jessie

  • Segregate container networks.

    • The default bridge network exists on all Docker hosts—if you do not specify a different network, new containers automatically connect to it.
    • Use custom bridge networks to control which containers can communicate between them, and to enable automatic DNS resolution from container name to IP address.
    • Ensure that containers can connect to each other only if absolutely necessary, and avoid connecting sensitive containers to public-facing networks.
    • Docker provides network drivers that let you create your own bridge network, overlay network, or macvlan network. If you need more control, you can create a Docker network plugin.

  • Improve container isolation.

Protecting a container is exactly the same as protecting any process running on Linux. Ideally, the operating system on a container host should protect the host kernel from container escapes, and prevent mutual influence between containers.

  • Set filesystem and volumes to Read only.

This can prevent malicious activity such as deploying malware on the container or modifying configuration. docker run --read-only alpine

  • Complete lifecycle management restrict system calls from Within Containers
  • Monitor Container Activity. Analyze collected events to detect suspicious behaviourial patterns.
  • Create an incident response process to ensure rapid response in the case of an attack.
  • Apply automated patching
  • Confirms Inmutability. Implement drift prevention to ensure container inmutability.
  • Ensure you have robust auditing and forensics for quick troubleshooting and compliance reporting.

Common Containers Attacks

Container Security Guides

Further reading:

Collaborate

If you find any typos, errors, outdated resources; or if you have a different point of view. Please open a pull request or contact me.

Pull requests and stars are always welcome 🙌