To cache images, Kubernetes Image Puller creates a Daemonset on the desired cluster, which in turn creates a pod on each node in the cluster consisting of a list of containers with command sleep 720h.
This ensures that all nodes in the cluster have those images cached. The sleep binary being used is golang-based (please see Scratch Images).
We also periodically check the health of the daemonset and re-create it if necessary.
The application can be deployed via Helm or by processing and applying OpenShift Templates. Also, there is a community supported operator available on the OperatorHub.
Configuration is done via env vars pulled from ./deploy/helm/templates/configmap.yaml, or ./deploy/openshift/configmap.yaml, depending on the deployment method.
The config values to be set are:
| Env Var | Usage | Default |
|---|---|---|
CACHING_INTERVAL_HOURS |
Interval, in hours, between checking health of daemonsets | "1" |
CACHING_MEMORY_REQUEST |
The memory request for each cached image when the puller is running | 10Mi |
CACHING_MEMORY_LIMIT |
The memory limit for each cached image when the puller is running | 20Mi |
CACHING_CPU_REQUEST |
The CPU request for each cached image when the puller is running | .05 or 50 millicores |
CACHING_CPU_LIMIT |
The CPU limit for each cached image when the puller is running | .2 or 200 millicores |
DAEMONSET_NAME |
Name of daemonset to be created | kubernetes-image-puller |
NAMESPACE |
Namespace where daemonset is to be created | kubernetes-image-puller |
IMAGES |
List of images to be cached, in this format <name>=<image>;... |
Contains a default list of images, but should be configured when deploying |
NODE_SELECTOR |
Node selector applied to pods created by the daemonset, provided in this format '{"key":"value"}' |
'{}' |
IMAGE_PULL_SECRETS |
List of image pull secrets, in this format pullsecret1;... to add to pods created by the DaemonSet. Those secrets need to be in the image puller's namespace and a cluster administrator must create them. |
"" |
AFFINITY |
Affinity applied to pods created by the daemonset, in this format '{"nodeAffinity":{ ... }}' |
'{}' |
KIP_IMAGE |
The image puller image to copy the sleep binary from |
quay.io/eclipse/kubernetes-image-puller:next |
TOLERATIONS |
Tolerations applied to pods created by the daemonset, provided in this format '[{"operator":"Exists"}]' |
'[]' |
The following values can be set:
| Value | Usage | Default |
|---|---|---|
deploymentName |
The value of DAEMONSET_NAME to be set in the ConfigMap, as well as the name of the deployment |
kubernetes-image-puller |
image.repository |
The repository to pull the image from | quay.io/eclipse/kubernetes-image-puller |
image.tag |
The image tag to pull | next |
serviceAccount.name |
The name of the ServiceAccount to create | k8s-image-puller |
configMap.name |
The name of the ConfigMap to create | k8s-image-puller |
configMap.images |
The value of IMAGES to be set in the ConfigMap |
// TODO create a reasonable set of default containers |
configMap.cachingIntervalHours |
The value of CACHING_INTERVAL_HOURS to be set in the ConfigMap |
"1" |
configMap.cachingMemoryRequest |
The value of CACHING_MEMORY_REQUEST to be set in the ConfigMap |
"10Mi" |
configMap.cachingMemoryLimit |
The value of CACHING_MEMORY_LIMIT to be set in the ConfigMap |
"20Mi" |
configMap.cachingCpuRequest |
The value of CACHING_CPU_REQUEST to be set in the ConfigMap |
.05 |
configMap.cachingCpuLimit |
The value of CACHING_CPU_LIMIT to be set in the ConfigMap |
.2 |
configMap.nodeSelector |
The value of NODE_SELECTOR to be set in the ConfigMap |
"{}" |
configMap.imagePullSecrets |
The value of IMAGE_PULL_SECRETS |
"" |
configMap.affinity |
The value of AFFINITY to be set in the ConfigMap |
"{}" |
configMap.tolerations |
The value of TOLERATIONS to be set in the ConfigMap |
"[]" |
The following values can be set:
| Parameter | Usage | Default |
|---|---|---|
SERVICEACCOUNT_NAME |
Name of service account used by main pod | k8s-image-puller |
IMAGE |
Name of image used for main pod | quay.io/eclipse/kubernetes-image-puller |
IMAGE_TAG |
Tag of image used for main pod | next |
DAEMONSET_NAME |
The value of DAEMONSET_NAME to be set in the ConfigMap |
"kubernetes-image-puller" |
DEPLOYMENT_NAME |
The name of the image puller deployment | "kubernetes-image-puller" |
CACHING_INTERVAL_HOURS |
The value of CACHING_INTERVAL_HOURS to be set in the ConfigMap |
"1" |
CACHING_MEMORY_REQUEST |
The value of CACHING_MEMORY_REQUEST to be set in the ConfigMap |
"10Mi" |
CACHING_MEMORY_LIMIT |
The value of CACHING_MEMORY_LIMIT to be set in the ConfigMap |
"20Mi" |
CACHING_CPU_REQUEST |
The value of CACHING_CPU_REQUEST to be set in the ConfigMap |
.05 |
CACHING_CPU_LIMIT |
The value of CACHING_CPU_LIMIT to be set in the ConfigMap |
.2 |
NAMESPACE |
The value of NAMESPACE to be set in the ConfigMap |
k8s-image-puller |
NODE_SELECTOR |
The value of NODE_SELECTOR to be set in the ConfigMap |
"{}" |
IMAGE_PULL_SECRETS |
The value of IMAGE_PULL_SECRETS |
"" |
AFFINITY |
The value of AFFINITY to be set in the ConfigMap |
"{}" |
TOLERATIONS |
The value of TOLERATIONS to be set in the ConfigMap |
"[]" |
kubectl create namespace k8s-image-puller
helm install kubernetes-image-puller -n k8s-image-puller deploy/helm
To set values, change deploy/helm/values.yaml or use --set property.name=value
OpenShift has a notion of project quotas to limit the aggregate resource consumption per project/namespace. The namespace that the image puller is deployed in must have enough memory and CPU to run each container for each node in the cluster:
(memory/CPU limit) * (number of images) * (number of nodes in cluster)
For example, running the image puller that caches 5 images on 20 nodes, with a container memory limit of 5Mi, your namespace would need a quota of 500Mi.
oc new-project k8s-image-puller
oc process -f deploy/openshift/serviceaccount.yaml | oc apply -f -
oc process -f deploy/openshift/configmap.yaml | oc apply -f -
oc process -f deploy/openshift/app.yaml | oc apply -f -
To change parameters, add -p PARAM=value to the oc process command, before piping to oc apply.
# Build Go binary:
make build
# Make docker image:
make docker
# The above:
make
# Clean:
make cleanThe provided Makefile has two parameters:
DOCKERIMAGE_NAME: name for docker imageDOCKERIMAGE_TAG: tag for docker image
Build:
CGO_ENABLED=1
BINARY_NAME=kubernetes-image-puller
GOOS=linux go build -v -o ./bin/${BINARY_NAME} ./cmd/main.go
GOOS=linux go build -a -ldflags '-w -s' -a -installsuffix cgo -o ./bin/sleep ./sleep/sleep.goMake docker image:
DOCKERIMAGE_NAME=kubernetes-image-puller
DOCKERIMAGE_TAG=next
docker build -t ${DOCKERIMAGE_NAME}:${DOCKERIMAGE_TAG} -f ./build/dockerfiles/Dockerfile .Once built and published to a registry, you can test FIPS compliance using https://github.com/openshift/check-payload#scan-a-container-or-operator-image
To run the unit tests:
make testEnd to end tests require kind.
Note that kind should not be installed with go get from this repository's directory.
cd $HOME && GO111MODULE="on" go get sigs.k8s.io/kind@v0.7.0 && cd ~-
./hack/run-e2e.shWill start a kind cluster and run the end-to-end tests in ./e2e. To remove the cluster after running the tests, pass the --rm argument to the script, or run kind delete cluster --name k8s-image-puller-e2e.
The image puller also supports pre-pulling the scratch images.
Previously the image puller was not able to pull scratch images, as they do not contain a sleep command.
However, the daemonset created by the Kubernetes Image Puller now:
- creates an
initContainerthat copies a golang-basedsleepbinary to a commonkipvolume. - creates containers
volumeMountsset to thekipvolume, and withcommandset to/kip/sleep 720h
As a result, every container (including scratch image containers) uses the provided golang-based sleep binary.
"Che" is a trademark of the Eclipse Foundation.