#enrober
This project consists of a an API server that functions as a wrapper around the kubernetes client library. The server can be deployed both locally and as a docker container within a kubernetes cluster.
This project is closely related to other 30x projects:
###Local Deployment
go build
./enroberThe server will be accesible at localhost:9000/
For the server to be able to communicate with your kubernetes cluster you must run:
kubectl proxy --port=8080 &
Please note that this allows for insecure communication with your kubernetes cluster and shuold only be used for testing.
###Kubernetes Deployment
A prebuilt docker image is available with:
docker pull thirtyx/enrober:v0.2.3To deploy the server as a docker container on a kubernetes cluster you should use the provided deploy-base.yaml file. Running kubectl create -f deploy-base.yaml will pull the image from dockerhub and deploy it to the default namespace.
The server will be accesible at <pod-ip>/
You can choose to expose the pod using the k8s-pods-ingress. Make sure to modify the deploy.yaml file to match your ingress configuration.
Alternatively you can expose the server using a kubernetes service. Refer to the docs here.
###Privileged Containers
By default enrober doesn't allow privileged containers to be deployed and will modify the containers security context at deploy time so that Priveleged = false. If you have a need for privileged containers set the ALLOW_PRIV_CONTAINERS environment variable to "true" in enrobers deployment yaml file.
##API Design
A swagger.yaml file is provided that documents the API per the OpenAPI specification.
##Key Components
####Environments
An environment consists of a kubernetes namespace and our specific secrets associated with it. Each environment comes with a routing secret that contains two key-value pairs, a public-api-key and a private-api-key. These are for use with the k8s-pods-ingress to allow for secure communication with pods from inside and outside of the kubernetes cluster.
##Apigee Specific Annotations
####Environments
When created environments can accept an array of valid host names to accept traffic from. This array is represented on the namespace object as a space delimited annotation. The individual values must be either a valid IP address or valid host name.
####Deployments
When created deployments can accept a publicHosts value, a privateHosts value or both. These values are for use with the k8s-pods-ingress and are the host name where the deployment can be reached. These values are stored as annotations on the deployed pods.
####Pod Template Specs
When they are provided to the deployments endpoint pod template specs must have several Apigee specific labels and annotations.
Labels:
For pods to be recognized by the k8s-pods-ingress they must have a label named "routable" with a value of "true".
Annotations:
For pods to be properly routed by the k8s-pods-ingress they must have a "publicPaths" and/or "privatePaths" annotation where the value is of the form {PORT}:{PATH}. You may have multiple space delimited {PORT}:{PATH} combinations on each annotation.
##Usage
This assumes you are running the server locally, it is accessible at localhost:9000, and your kubernetes cluster is exposed with
kubectl proxy --port=8080
Note: All API calls require a valid JWT to be passed into an authorization header. For these examples we are using an empty JSON object that has been base64 encoded.
####Create a new environment:
curl -X POST -d '{
"environmentName": "org1-env1",
"hostNames": ["host1"]
}' \
"localhost:9000/environments"This will create a org1-env1 namespace and a secret named routing with two key-value pairs:
- public-api-key
- private-api-key
The value of each of these keys-value pairs will a 256-bit base64 encoded randomized string. These secrets are for use with 30x/k8s-pods-ingress
###Update the environment
curl -X PATCH -d '{
"hostNames": ["host1", "host2"]
}' \
"localhost:9000/environments/org1-env1"This will modify the previously created environment's hostNames array to equal:
["host1", "host2"]
curl -X POST -d '{
"deploymentName": "dep1",
"publicHosts": "deploy.k8s.public",
"privateHosts": "deploy.k8s.private",
"replicas": 1,
"pts":
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"name": "nginx-and-helloworld",
"labels": {
"app": "web",
},
"annotations": {
"publicPaths": "80:/ 90:/2",
"privatePaths": "80:/ 90:/2"
}
},
"spec": {
"containers": [{
"name": "nginx",
"image": "nginx",
"env": [{
"name": "PORT",
"value": "80"
}],
"ports": [{
"containerPort": 80
}]
}, {
"name": "test",
"image": "jbowen/testapp:v0",
"env": [{
"name": "PORT",
"value": "90"
}],
"ports": [{
"containerPort": 90
}]
}]
}
},
"envVars": [{
"name": "test1",
"value": "test3"
},
{
"name": "test2",
"value": "test4"
}]
}' \
"localhost:9000/environments/org1-env1/deployments"This will create a deployment that will guarantee a single replica of a pod consisting of two containers:
- An nginx container serving on port 80
- A hello world container serving on port 90
The deployed pod will have also two environment variables:
- test1=test3
- test2=test4
curl -X PATCH -d '{
"replicas": 3,
}' \
"localhost:9000/environments/org1-env1/deployments/dep1"This will modify the previous deployment to now guarantee 3 replicas of the pod.
Note that if you update a deployment without passing in the
"replicas"parameter it will default to 0 replicas.
###Delete deployment
curl -X DELETE \
"localhost:9000/environments/org1-env1/deployments/dep1"This will delete the previously created deployment and all related resources such as replica sets and pods.
###Delete environment
curl -X DELETE \
"localhost:9000/environments/org1-env1"This will delete the previously created environment.