container_cpu_detection

About

This program is used to autuomatically calculate the number of available CPU cores in the container based on the cgroup information by hijacking the sysconf system call. (This method of hijacking uses a Unix system trick LD_PRELOAD to hijack dynamic link library)

I have known the secnario of the usage is that Nginx calculates the number of available CPU cores by sysconf(_SC_NPROCESSORS_ONLN) and starts the corresponding number of worker processes. In addition, there must be many others usage scenarios.

Algorithm

There are three ways that Docker limits CPU resources.

cpuset
cpu quota & cpu period
cpu shares

Docker mounts cgroup information into container starting with version 1.8, so it is possible to determine the appropriate number of CPUs based on the cgroup information in the container.

Assume that the cgroup information is mounted in the /sys/fs/cgroup/:

Step 1 - Get the value of the `cpuset`

By reading /sys/fs/cgroup/cpuset/cpuset.cpus. By default, this value is equal to the number of the host CPU cores.

Step 2 - Get the value of the `cpu quota & cpu period`

By reading /sys/fs/cgroup/cpu/cpu.cfs_period_us & /sys/fs/cgroup/cpu/cpu.cfs_quota_us. Calculate the available CPU cores by quota divided by period and rounded up.

Step 3 - Get the value of the `cpu shares`

By reading /sys/fs/cgroup/cpu/cpu.shares. If you don't specify the cpu shares when running a container then its default value is 1024.

Step 4 - Calculate the number of available CPU cores.

These three ways can be specified at the same time. When they are specfied at the same time, take the minimum value as the number of available CPU cores.

Usage

make
make test (Please make sure that docker has already installed.)

Note:

Set the environment variable DETECTION_TARGETS in the container to specify which programs you want to hijack. Multiple name can be specified at the same time. (separated by a colon)
Set the environment variable LD_PRELOAD in the container to specify the path of the .so file.
Running a container with -v parameter to mount .so file into the container.
Only for Linux.

For example:

docker run -ti --rm \
--cpuset-cpus 0,1 --cpu-quota 200000 \
-v `pwd`/detection.so:/usr/lib/detection.so \
-v `pwd`/sysconf_test:/tmp/sysconf_test \
-e DETECTION_TARGETS=sysconf_test \
-e LD_PRELOAD=/usr/lib/detection.so \
ubuntu /tmp/sysconf_test

agile6v/container_cpu_detection