Kubernetes is a platform for running and managing application containers and has slowly evolved into a platform for building platforms, largely thanks to its extensible API. Kubernetes has many extension points including extensions that enable you to define custom resource types, cloud provider and container runtime integrations.
Another, less well known, set of extension points are the admission controllers. An admission controller is a piece of code that intercepts requests to the Kubernetes API prior to persistence of the object, but after the request is authenticated and authorized. Most admission controllers are built into Kubernetes and cover a range of functionality.
To understand how admission controllers work you need to see them in action. Take the namespace exists admission controller for example, it rejects all requests that attempt to create resources in a namespace that does not exist.
If you were to list the active namespaces available to a new Kubernetes install you would see two or three namespaces including the default and kube-system namespaces. The kube-system namespace is where things like Kubernetes DNS and the Kubernetes Dashboard live.
kubectl get ns
output:
NAME STATUS AGE
default Active 3m
kube-public Active 3m
kube-system Active 3m
If you tried to create a deployment in a namespace that did not exist you would get an error because the namespace exists admission controller would reject it.
kubectl run nginx --image nginx --namespace does-not-exist
output:
Error from server (NotFound): namespaces "does-not-exist" not found
With a basic understanding of how admission controllers work it's time to take a look at the newest admission controller, the validating admission webhook. As it names implies, the validating admission webhook allows you to intercept and validate requests to the Kubernetes API using an external webhook, but not mutate them. That last point is critical; because validating admission webhooks can't mutate resources it's safe to run them in parallel, and quickly reject a request if any of the webhooks fail.
Before you can use them you'll need access to a Kubernetes cluster.
Before you can use validating admission webhooks you need access to a Kubernetes cluster.
There are many ways to provision a Kubernetes cluster, but I'm going to assume you want me to tell you exactly what to do so we can get back to building a validating admission webhook. This is where I point you to Google Kubernetes Engine and swear it's not a vendor pitch. While I'm recommending GKE, everything should work on minikube or Docker support for Kubernetes.
If you've chosen to follow along with GKE, run the following commands and wait for them to finish.
First create a 1.9.2+ Kubernetes cluster. We are going to spin a 1 node cluster to help save you some cash, and lighten the blow those bitcoin crashes are taking on your wallet.
Use the gcloud command to create the k0 Kubernetes cluster:
gcloud container clusters create k0 \
--cluster-version 1.9.2-gke.1 \
--zone us-central1-a \
--num-nodes 1
With the k0 cluster in place it's time to design and build a validating admission webhook.
We are going to write our first validating admission webhook using nocode.
I'll give you a minute.
That's right, don't write a single line of code until you determine what rules you plan to use for accepting or rejecting a Kubernetes resource. For our first webhook we are going to keep things simple. We are going to reject all pods that leverage environment variables for application configuration.
You can write validating admission webhooks in just about any programming language, except nocode, and deploy it on any platform, including Kubernetes. But remember, the only goal is to validate incoming Kubernetes resources, not build a generic web application. In theory we only need to write a small bit of code to make that happen.
function denyenv (req, res) {
// Review the Kubernetes Pod resource and reject it if any
// of the containers are using environment variables.
}Given the minimual requirements for our validating admission webhook, we are going to skip containers altogether and deploy our webhook using a Function as a Service (FaaS) platform. Also know as Serverless.
We are going to implement our validating admission webhook using Node.js and deploy it to Google Cloud Functions.
Now it's time to write the validating admission webhook. Create a directory named denyenv and move into it:
mkdir denyenv && cd denyenv
Next, save the following code block to a file named index.js:
'use strict';
exports.denyenv = function denyenv (req, res) {
var admissionRequest = req.body;
// Get a reference to the pod spec
var object = admissionRequest.request.object;
console.log(`validating the ${object.metadata.name} pod`);
var admissionResponse = {
allowed: false
};
var found = false;
for (var container of object.spec.containers) {
if ("env" in container) {
console.log(`${container.name} is using env vars`);
admissionResponse.status = {
status: 'Failure',
message: `${container.name} is using env vars`,
reason: `${container.name} is using env vars`,
code: 402
};
found = true;
};
};
if (!found) {
admissionResponse.allowed = true;
}
var admissionReview = {
response: admissionResponse
};
res.setHeader('Content-Type', 'application/json');
res.send(JSON.stringify(admissionReview));
res.status(200).end();
};Run the following command to create a new function named denyenv:
gcloud beta functions deploy denyenv --trigger-http
Retrieve the HTTPS URL that triggers the denyenv function as we'll need it later when configuring our Kubernetes cluster to use it.
HTTPS_TRIGGER_URL=$(gcloud beta functions describe denyenv \
--format 'value(httpsTrigger.url)')
With the denyenv function in place, it’s time to configure the Kubernetes cluster to use it.
Start by generating a validating webhook configuration and submitting it to the Kubernetes API:
cat <<EOF | kubectl apply -f -
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
name: denyenv
webHooks:
- name: denyenv.hightowerlabs.com
rules:
- apiGroups:
- ""
apiVersions:
- v1
operations:
- CREATE
resources:
- pods
failurePolicy: Fail
clientConfig:
url: "${HTTPS_TRIGGER_URL}"
EOF
output:
validatingwebhookconfiguration "denyenv" created
At this point we are all set to start testing the denyenv webhook.
With the the denyenv validating admission webhook in place we need to ensure that it’s working.
First let’s make sure pods without env vars can be deployed to our Kubernetes clusters:
kubectl run nginx --image=nginx
output:
deployment "nginx" created
List the pods:
kubectl get pods
output:
NAME READY STATUS RESTARTS AGE
nginx-8586cf59-qmdm4 1/1 Running 0 25s
It works. We can review the logs of the denyenv cloud function to show that it was indeed called:
gcloud beta functions logs read denyenv
output:
D denyenv wb9bmjb34uyh 2018-02-09 02:06:35.534 Function execution started
I denyenv wb9bmjb34uyh 2018-02-09 02:06:35.542 validating the nginx-8586cf59-qmdm4 pod
D denyenv wb9bmjb34uyh 2018-02-09 02:06:35.545 Function execution took 12 ms, finished with status code: 200
Next we need to ensure our webhook actually does the thing it’s designed to do, which is prevent pods with containers using env vars.
kubectl run nginx-with-env --image=nginx --env="PASSWORD=fail"
output:
deployment "nginx-with-env" created
List the pods:
kubectl get pods
output:
NAME READY STATUS RESTARTS AGE
nginx-8586cf59-qmdm4 1/1 Running 0 1m
Its not there. What's going on?
The best way to determine why the nginx-with-env pod is not running is to review the Kubernetes event stream:
kubectl get events
output:
Warning FailedCreate replicaset-controller Error creating: admission webhook "denyenv.hightowerlabs.com" denied the request: nginx-with-env is using env vars
Looks like the nginx-with-env pod was denied by the denyenv admission webhook. We can confirm by fetching the logs for the denyenv function.
gcloud beta functions logs read denyenv
I denyenv s1ij4wfykerr 2018-02-09 02:09:31.457 validating the nginx-with-env-76498b5b66-nw259 pod
I denyenv s1ij4wfykerr 2018-02-09 02:09:31.459 nginx-with-env is using env vars
D denyenv s1ij4wfykerr 2018-02-09 02:09:31.461 Function execution took 8 ms, finished with status code: 200
Validating admission webhooks are one of easiest ways of extending Kubernetes with new policy controls. Building and running admission webhooks using a FaaS platform can help streamline the development process, and make it easy to enforce policy across multiple Kubernetes clusters using a single function.