A set of scripts for provisioning and running Kubernetes on a Raspberry Pi cluster.
WARNING: As this cluster is intended to run locally, certain security aspects have been ignored. This setup is NOT intended to be ran in a production environment!
- Download the Raspbian Lite image from the Raspberry Pi website.
- Insert SD card and run
lsblk
to get block device name. Eg./dev/sda
. - Run ./flash.sh script and provide the relevant paths and config.
- Boot Pi from the flashed SD card.
- SSH onto the Pi:
ssh pi@[IP address]
. cd
into$HOME/bin
.- Provision node using:
./bootstrap.sh
- Note: This script will reboot the Pi. - Install required packages using:
./packages.sh
. - Initialise master node using:
./master.sh
.
Once all scripts have ran successfully, validate the master node is showing a "Ready" status: kubectl get nodes
.
- Boot Pi from the flashed SD card.
- SSH onto the Pi:
ssh pi@[IP address]
. cd
into$HOME/bin
.- Provision node using:
./bootstrap.sh
. NOTE: This script will reboot the Pi. - Install required packages using:
./packages.sh
. - Log out.
After provisioning the new node, you will need to generate a new kubeadm join token:
- SSH onto the master node:
ssh pi@[IP address]
. - Create a new token:
kubeadm token create --print-join-command
. - Log out.
Once you have the new token and hash:
- SSH onto the new worker node.
- Run the
kubeadm join
command generated on the master node.
Once the worker node has successfully joined the cluster, validate the worker node is showing a "Ready" status: kubectl get nodes
.
Several yaml files have been provided to make it easier to configure some core monitoring applications for the cluster. In most cases, these yaml files are direct copies of the default yaml files provided by the owner, with a change to use an ARM image.
Deploy the dashboard using:
kubectl apply -f spec/kubernetes/dashboard/dashboard.yml
Check that the pod is running:
kubectl get pods -n=kube-system | grep "dashboard"
Once running, proxy the API server and visit the dashboard:
kubectl proxy
More specific information can be found in the official documentation.
Deploy the metrics generator using:
kubectl apply -f spec/kubernetes/kube-state-metrics/kube-state-metrics.yml
More specific information can be found in the official documentation.
Deploy the metrics server using:
kubectl apply -f spec/kubernetes/metrics-server/metrics-server.yml
More specific information can be found in the official documentation.
Note: For simplicity, the included Prometheus yaml uses a Deployment instead of a StatefulSet. Therefore, any persisting data or changes made to the config via the UI, will be deleted if the pod is to restart.
Deploy Prometheus:
kubectl apply -f spec/prometheus/prometheus.yml
Check that the pod is running:
kubectl get pods | grep "prometheus"
Once running, port-forward the pod and visit the dashboard:
kubectl port-forward prometheus-xxxx-xxx 9090
By default, several scrape-configs have been included for the followings jobs:
kubernetes-apiservers
kubernetes-nodes
kubernetes-cadvisor
kubernetes-service-endpoints
kubernetes-services
kubernetes-pods
You should be able to query metrics after 30s.
More specific information can be found in the official documentation.
Note: For simplicity, the included Grafana yaml uses a Deployment instead of a StatefulSet. Therefore, any persisting data or changes made to the config via the UI, will be deleted if the pod is to restart.
Deploy Grafana:
kubectl apply -f spec/grafana/grafana.yml
Check that the pod is running:
kubectl get pods | grep "grafana"
Once running, port-forward the pod and visit the dashboard:
kubectl port-forward granana-xxxx-xxx 3000
By default, there is a pre-configured datasource for Prometheus. There are also several dashboards in spec/grafana/dashboards/
that can be manually imported.
More specific information can be found in the official documentation.
Alongside the provided yaml files, a few other tools that are incredibly useful to run in a Kubernetes cluster are:
- Linkerd: a service mesh, designed to give platform-wide observability, reliability, and security.