dongcarl/Docker-Secure-Deployment-Guidelines

Deployment checklist for securely deploying Docker

Docker Secure Deployment Guidelines

Within today’s growing cloud-based IT market, there is a strong demand for virtualisation technologies. Unfortunately most virtualisation solutions are not flexible enough to meet developer requirements and the overhead implied by the use of full virtualisation solutions becomes a burden on the scalability of the infrastructure.

Docker reduces that overhead by allowing developers and system administrators to seamlessly deploy containers for applications and services required for business operations. However, because Docker leverages the same kernel as the host system to reduce the need for resources, containers can be exposed to significant security risks if not adequately configured.

The following itemised list suggests hardening actions that can be undertaken to improve the security posture of the containers within their respective environment. It should be noted that proposed solutions only apply to deployment of Linux Docker containers on Linux-based hosts, using the most recent release of Docker at the time of this writing (1.4.0, commit 4595d4f, dating 11/12/14).

Part of the content below is based on publications from Jérôme Petazzoni ^[1] and Daniel J Walsh ^[2]. This document aims at adding on to their recommendations and how they can specifically be implemented within Docker.

Note: Most of suggested command line options can be stored and used in a similar manner inside a Dockerfile for automated image building.

Item	Deployment
Docker Images	Docker 1.3 now supports cryptographic signatures [3] to ascertain the origin and integrity of official repository images. This feature is however still a work in progress as Docker will issue a warning but not prevent the image from actually running. Furthermore, it does not apply to non-official images. In general, ensure that images are only retrieved from trusted repositories and that the --insecure-registry=[] command line option is never used.
Network Namespaces [4]	By default, the Docker REST API used to control containers exposed via the system Docker daemon is only accessible locally via a Unix domain socket. Running Docker on a TCP port (i.e. forcing the bind address using the -H option when launching the Docker daemon) will allow anyone with access to that port to gain access to the container, potentially gaining root access on the host as well in some scenarios where the local user belongs to the docker group [5]. When allowing access to the daemon over TCP, ensure that communications are adequately encrypted using SSL [6] and access controls effectively prevent unauthorised parties from interacting with it. Kernel firewall iptables rules can be applied to docker0, the standard network bridge interface for Docker, to enforce those controls. For instance, the source IP range of a Docker container can be restricted from talking with the outside world using the following iptables filter [7]. iptables -t filter -A FORWARD -s <source_ip_range> -j REJECT --reject-with icmp-admin-prohibited
Logging & Auditing	Collect and archive security logs relating to Docker for auditing and monitoring purposes. Accessing log files outside of the container, from the host [8], can be performed using the following command: docker run -v /dev/log:/dev/log <container_name> /bin/sh Using the Docker command built-in: docker logs ... (-f to follow log output) Log files can also be exported for persistent storage into a tarball using: docker export ...
SELinux or AppArmor	Linux kernel security modules such as Security-Enhanced Linux (SELinux) and AppArmor can be configured, via access control security policies, to implement mandatory access controls (MAC) confining processes to a limited set of system resources or privileges. SELinux can be enabled in the container using setenforce 1, if it was previously installed and configured. The SELinux support for the Docker daemon is disabled by default and needs to be enabled using --selinux-enabled. Introduced in Docker version 1.3 [9], label confinement for the container can be configured using the newly added --security-opt argument to load SELinux or AppArmor policies, as shown in the Docker run reference excerpt below. --security-opt="label:user:USER" : Set the label user for the container --security-opt="label:role:ROLE" : Set the label role for the container --security-opt="label:type:TYPE" : Set the label type for the container --security-opt="label:level:LEVEL" : Set the label level for the container --security-opt="apparmor:PROFILE" : Set the apparmor profile to be applied to the container Example: docker run --security-opt=label:level:s0:c100,c200 -i -t centos bash
Daemon Privileges	Do not use the --privileged command line option. This would otherwise allow the container to access all devices on the host and would in addition provide the container with specific a LSM (i.e SELinux or AppArmor) configuration that would give it the same level of access as processes running on the host. Avoiding the use of --privileged helps reduce the attack surface and potential of host compromise. This however does not mean that the daemon will run without root privileges which is still currently required in the latest release. The ability to launch the daemon and containers should only be given to trusted user. Minimize privileges enforced inside the container by leveraging the -u option. Example: docker run -u <username> -it <container_name> /bin/bash Any user part of the docker group could eventually get root on the host from the container
cgroups [10]	In order to prevent Denial of Service (DoS) attacks via system resource exhaustion, a number of resource restrictions can be applied using specific command line arguments. CPU usage: docker run -it --rm --cpuset=0,1 -c 2 ... Memory usage: docker run -it --rm -m 128m ... Storage usage: docker -d --storage-opt dm.basesize=5G Disk I/O: Currently not supported by Docker. BlockIO* properties exposed via systemd can be leveraged to control disk usage quotas on supported operating systems.
SUID/GUID binaries	SUID and GUID binaries can prove dangerous when vulnerable to attacks leading to arbitrary code execution (e.g. buffer overflows), as they will be running under the context of the process’s file owner or group. When possible, prohibit SUID and SGID from taking effect by reducing the capabilities given to containers using specific command line arguments. docker run -it --rm --cap-drop SETUID --cap-drop SETGID ... Alternatively, consider removing SUID capabilities from the system by mounting filesystem with the nosuid attribute. One last option could be to remove unwanted SUID and GUID binaries from the system altogether. These types of binaries can be found on a Linux system by running the following commands: find / -perm -4000 -exec ls -l {} \; 2>/dev/null find / -perm -2000 -exec ls -l {} \; 2>/dev/null The SUID and GUID file permissions can then be removed using commands similar to the following [11]: sudo chmod u-s filename sudo chmod -R g-s directory
Devices control group (/dev/*)	If required, mount devices using the built-in --device option (do not use -v with the --privileged argument). Granular permissions can be assigned to each device using a third set of options :rwm to override read, write, and mknod permissions respectively (default includes all permissions). Example (for using sound card with read-only permission): docker run --device=/dev/snd:/dev/snd:r ... This feature was introduced in version 1.2 [12].
Services and Applications	To reduce the potential for lateral movement if a Docker container was to be compromised, consider isolating sensitive services (e.g. run SSH service on bastion host or in a VM). Furthermore, do not run untrusted applications with root privileges within containers.
Mount Points	This is handled automatically by Docker when using the native container library (i.e. libcontainer). However, when using the LXC container library, sensitive mount points should ideally be manually mounted with read-only permissions, including: /sys /proc/sys /proc/sysrq-trigger /proc/irq /proc/bus Mount permissions should later be removed to prevent remounting.
Linux Kernel	Ensure kernel is up-to-date using update utility provided by the system (e.g. apt-get, yum, etc). Out-dated kernels are more likely to be vulnerable to publicly disclosed vulnerabilities. Use strengthened a kernel with GRSEC or PAX, that for example provide increased security against memory corruption bugs.
User Namespaces	Docker does not support user namespaces but is a feature currently under development [13]. UID mapping is currently supported by the LXC driver but not in the native libcontainer library. This feature would allow the Docker daemon to run as an unprivileged user on the host but appear as running as root within containers. While suing the LXC driver this can by using the -lxc-conf option, as shown in the example Docker run command below. docker run -lxc-conf="lxc.id_map = u 0 100000 65536" -lxc-conf="lxc.id_map = g 0 100000 65536" ... The specified arguments in the above command will instruct Docker to map UIDs and GIDs 100000 through 65536 on the host to UIDs and GIDs 0 through 65536 to a specific user account, defined at system level on the host in a configuration similar to: [14] /etc/subgid:<username>:100000:65536 /etc/subuid:<username>:100000:65536
libseccomp (and seccomp-bpf extension)	The libseccomp library allows restricting the use of Linux kernel’s syscall procedures based on a white-list approach. Syscall procedures not vital to system operation should ideally be disabled to prevent abuse or misuse within a compromised container. This feature is currently a work in progress (available in LXC driver, not in libcontainer which is now default). To restart the Docker daemon to use the LXC driver use [15]: docker -d -e lxc Instructions on how to generate a seccomp configuration are on the Docker GitHub repository within the 'contrib' [16] folder. This can later be used to create a LXC based Docker container using the following command: docker run --lxc-conf="lxc.seccomp=$file" <rest of arguments>
capabilities(7)	Drop linux capabilities to a minimum whenever possible. Docker default capabilities include: chown, dac_override, fowner, kill, setgid, setuid, setpcap, net_bind_service, net_raw, sys_chroot, mknod, setfcap, and audit_write. Can be controlled when launching a container from command line with --cap-add=[] or --cap-drop=[]. Example: docker run --cap-drop setuid --cap-drop setgid -ti <container_name> /bin/sh This feature was introduced in Docker version 1.2 [17]
Multi-tenancy Environments	Due to the shared nature of Docker containers’ kernel, separation of duty in multi-tenancy environments cannot be achieved securely. It is recommended that containers be run on hosts that have no other purposes and are not used for sensitive operations. Consider moving all services into containers controlled by Docker. When possible, keep inter-container communications to a minimum by setting the Docker daemon to use --icc=false and specify -link with docker run when necessary, or --export=port to expose a port from the container without publishing it on the host. Map groups of mutually-trusted containers to separate machines [18].
Full Virtualisation	Use a full virtualisation solution to contain Docker, such as KVM. This will prevent escalation from the container to the host if a kernel vulnerability is exploited inside the Docker image. Docker images can be nested to provide this KVM virtualisation layer as shown in the Docker-in-Docker utility [19].
Security Audits	Perform regular security audits of your host system and containers to identify mis-configuration or vulnerabilities that could expose your system to compromise.

References

[1] Docker, Linux Containers (LXC), and security (August, 2014). Jérôme Petazzoni. [presentation slides] http://www.slideshare.net/jpetazzo/docker-linux-containers-lxc-and-security
[2] Docker and SELinux (July, 2014). Daniel Walsh [video] https://www.youtube.com/watch?v=zWGFqMuEHdw
[3] Docker 1.3: Signed Images, Process Injection, Security Options, Mac shared directories (October, 2014). Scott Johnston http://blog.docker.com/2014/10/docker-1-3-signed-images-process-injection-security-options-mac-shared-directories/
[4] Exploring LXC Networking (November, 2013). Milos Gajdos. http://containerops.org/2013/11/19/lxc-networking/
PaaS under the hood, episode 1: kernel namespaces (November, 2012). Jérôme Petazzoni. http://blog.dotcloud.com/under-the-hood-linux-kernels-on-dotcloud-part
Exploring networking in Linux containers (January, 2014). Milos Gajdos. [presentation slides] https://speakerdeck.com/gyre007/exploring-networking-in-linux-containers
[5] How to grant rights to users to use Docker in Fedora (October 2014). Daniel Walsh http://opensource.com/business/14/10/docker-user-rights-fedora
[6] Running Docker with https. [Docker documentation] https://docs.docker.com/articles/https/
[7] security suggestions when running malicious code, Google Groups (August, 2013). Jérôme Petazzoni https://groups.google.com/forum/#!msg/docker-user/uuiQ3Nk3uSY/SuFpdO6BPmYJ
[8] Monitoring Images and Containers. [Red Hat documentation] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Resource_Management_and_Linux_Containers_Guide/sec-Monitoring_Images.html
[9] Docker 1.3: Signed Images, Process Injection, Security Options, Mac shared directories (October, 2014). Scott Johnston http://blog.docker.com/2014/10/docker-1-3-signed-images-process-injection-security-options-mac-shared-directories/
[10] Resource management in Docker (September, 2014). Marek Goldmann. https://goldmann.pl/blog/2014/09/11/resource-management-in-docker/
Gathering LXC and Docker Containers Metrics (October, 2013). Jérôme Petazzoni. http://blog.docker.com/2013/10/gathering-lxc-docker-containers-metrics/
[11] Removing SUID and SGID flags off binaries (August, 2008). Eric Thern. http://www.thern.org/projects/linux-lecture/intro-to-linux/node10.html
[12] Announcing Docker 1.2.0 (August, 2014). Victor Vieux. http://blog.docker.com/2014/08/announcing-docker-1-2-0/
[13] Having non-root privileges on the host and root inside the container #2918 (November, 2013). [GitHub issue] moby/moby#2918
Support for user namespaces #4572 (March 2014). [GitHub issue] moby/moby#4572
Proposal: Support for user namespaces #7906 (September, 2014). [GitHub issue] moby/moby#7906
Issue 8447: syscall, os/exec: Support for User Namespaces (July, 2014) [Google Code issue] https://code.google.com/p/go/issues/detail?id=8447
[14] Introduction to unprivileged containers (January, 2014). Stéphane Graber https://www.stgraber.org/2014/01/17/lxc-1-0-unprivileged-containers/
[15] Docker 0.9: Introducing Execution Drivers and libcontainer (March, 2014). Solomon Hykes http://blog.docker.com/2014/03/docker-0-9-introducing-execution-drivers-and-libcontainer/
[16] A simple helper script to help people build seccomp profiles for Docker/LXC (November 2013). Martijn van Oosterhout. https://github.com/docker/docker/blob/487a417d9fd074d0e78876072c7d1ebfd398ea7a/contrib/mkseccomp.pl
https://github.com/docker/docker/blob/487a417d9fd074d0e78876072c7d1ebfd398ea7a/contrib/mkseccomp.sample
[17] Announcing Docker 1.2.0 (August, 2014). Victor Vieux. http://blog.docker.com/2014/08/announcing-docker-1-2-0/
[18] Docker Container Breakout Proof-of-Concept Exploit (June, 2014). James Turnbull http://blog.docker.com/2014/06/docker-container-breakout-proof-of-concept-exploit/
[19] docker2docker GitHub repository. Jérôme Petazzoni. https://github.com/jpetazzo/docker2docker

License

Docker Secure Deployment Guidelines by Gotham Digital Science is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.