kube-spawn is a tool for creating a multi-node Kubernetes cluster on a single Linux machine, created mostly for developers of Kubernetes but is also a Certified Kubernetes Distribution and, therefore, perfect for running and testing deployments locally.
It attempts to mimic production setups by making use of OS containers to set up nodes.
-
Host:
systemd-nspawnat least version 233qemu-img
-
Kubernetes at least version 1.7.0
kube-spawn should run well on Fedora 26. If you want to test it in a
controlled environment, you can use Vagrant.
To setup kube-spawn on your machine, make sure you have a working Go environment:
# Get CNI plugins
$ go get -u github.com/containernetworking/plugins/plugins/...
# Get the source for this project
$ go get -d github.com/kinvolk/kube-spawn
kube-spawn will configure the networks it needs in /etc/cni/net.d.
# Build the tool
$ cd $GOPATH/src/github.com/kinvolk/kube-spawn
$ make all
# (optional) Install binaries under a system directory.
# The install prefix defaults to /usr, which you can override with an env
# variable $PREFIX, like "make PREFIX=/usr/local install".
$ sudo make install
$ export CNI_PATH=$GOPATH/bin
# This generated a default 3 nodes cluster configuration
$ sudo -E ./kube-spawn create --nodes=3
$ sudo -E ./kube-spawn start
The create subcommand sets up a cluster environment in /var/lib/kube-spawn, and puts all the neccessary
scripts/configs into place for running the cluster.
Via start you bring up the nodes and kube-spawn configures the cluster using kubeadm.
Now that you're up and running, you can start using it.
After stopping the cluster with stop you don't have to run create again, unless you want to change the cluster config in
/var/lib/kube-spawn/CLUSTER_NAME/kspawn.toml.
kube-spawn creates the config at /var/lib/kube-spawn/default. There you can find the kubeconfig for the cluster and a token file with a kubeadm token, that can be used to join more nodes.
To verify everything worked you can run:
$ export KUBECONFIG=/var/lib/kube-spawn/default/kubeconfig
$ kubectl get nodes
$ kubectl get pods --all-namespaces
$ kubectl create -f 'https://raw.githubusercontent.com/kubernetes/kubernetes/master/examples/guestbook/all-in-one/frontend.yaml'
If you don't have kubectl, you can get it with:
KUBERNETES_VERSION=$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)
sudo curl -Lo /usr/local/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/${KUBERNETES_VERSION}/bin/linux/amd64/kubectl
sudo chmod +x /usr/local/bin/kubectl
It is possible to run rktlet on kube-spawn. See doc/rktlet.
One of the main use cases of kube-spawn is to be able to easily test patches to
Kubernetes. To do this, some additional steps are required.
To get the Hyperkube image of your local Kubernetes build to deploy, kube-spawn sets up
a local insecure Docker registry. Pushing images to it needs to be enabled by adding
the following to the docker daemon configuration file (/etc/docker/daemon.json).
...
"insecure-registries": [
"10.22.0.1:5000"
]
...
The following steps assume you have a local checkout of the Kubernetes source.
# Build Kubernetes
$ cd $GOPATH/src/k8s.io/kubernetes
$ build/run.sh make
# Build a Hyperkube Docker image
$ git describe
v1.8.5-beta.0
$ make -C cluster/images/hyperkube VERSION=v1.8.5-beta.0-myfeature
This will create a Docker image with a name gcr.io/google-containers/hyperkube-amd64:v1.8.5-beta.0-myfeature.
To check if it is created correctly, do so:
$ docker images | grep hyperkube-amd64
gcr.io/google-containers/hyperkube-amd64 v1.8.5-beta.0-myfeature 8687537eff68 10 minutes ago 530 MB
Assuming you have built kube-spawn and pulled the CoreOS image, do:
# Spawn and provision nodes for the cluster
$ sudo -E ./kube-spawn create --dev -t v1.8.5-beta.0-myfeature -c myfeature
$ sudo -E ./kube-spawn start -c myfeature
For a specific example, see doc/dev-workflow.
All nodes can be seen with machinectl list, machinectl shell can be used to access a node, for example:
sudo machinectl shell root@kubespawn0
The password is k8s.
Run kube-spawn -h