This is a Kubernetes configuration for provisioning a stack suitable for displaying build statistics about the Zephyr RTOS.
Development is being done with Windows 10 and Rancher Desktop (Docker mode). However, the work should apply to any Kubernetes distribution.
Follow the official installation instructions to install the latest minikube release for Linux x86-64 using the stable version via binary download.
$ minikube dashboard
To access the Kubernetes dashboard from other machines on your network set up the proxy to listen on all interfaces and accept all hosts:
$ kubectl proxy --address='0.0.0.0' --accept-hosts='^*$'
Starting to serve on [::]:8001
You should now be able to access the dashboard at your machine's external IP address: http://192.168.10.17:8001/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/
Run minikube tunnel to create external IPs for the services.
$ minikube tunnel
Verify that the EXTERNAL-IP column goes from (pending) to a valid IP.
$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
chronograf LoadBalancer 10.99.89.126 10.99.89.126 8888:32418/TCP 73m
grafana LoadBalancer 10.109.155.193 10.109.155.193 3000:31700/TCP 73m
influxdb LoadBalancer 10.102.50.95 10.102.50.95 8086:31324/TCP 73m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h
To see all services that are running use --all with the services command:
$ minikube service --all
* service default/kubernetes has no node port
|-----------|------------|--------------|---------------------------|
| NAMESPACE | NAME | TARGET PORT | URL |
|-----------|------------|--------------|---------------------------|
| default | chronograf | 8888 | http://192.168.49.2:32418 |
| default | grafana | 3000 | http://192.168.49.2:31700 |
| default | influxdb | 8086 | http://192.168.49.2:31324 |
| default | kubernetes | No node port |
|-----------|------------|--------------|---------------------------|
By design, Minikube only exposes Kubernetes services to the host machine. In order to access the grafana service from a remote machine you must create an iptables rule to NAT packets into the minikube VM.
MINIKUBE_IP=$(minikube ip)
GRAFANA_PORT=$(minikube service --url grafana | cut -d: -f3)
echo "${MINIKUBE_IP} ${GRAFANA_PORT}"
sudo iptables -t nat -A DOCKER ! -i docker0 -p tcp -j DNAT --dport 3000 --to-destination "${MINIKUBE_IP}:${GRAFANA_PORT}"
This adds the last rule show below in the DOCKER chain.
$ sudo iptables -t nat -L DOCKER
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere
RETURN all -- anywhere anywhere
RETURN all -- anywhere anywhere
DNAT tcp -- anywhere localhost tcp dpt:49153 to:192.168.49.2:32443
DNAT tcp -- anywhere localhost tcp dpt:49154 to:192.168.49.2:8443
DNAT tcp -- anywhere localhost tcp dpt:49155 to:192.168.49.2:5000
DNAT tcp -- anywhere localhost tcp dpt:49156 to:192.168.49.2:2376
DNAT tcp -- anywhere localhost tcp dpt:49157 to:192.168.49.2:22
DNAT tcp -- anywhere anywhere tcp dpt:3000 to:192.168.49.2:31700
You should now be able to access the Grafana dashboard here: http://192.168.10.17:3000/
Optionally, if you need to expose influxdb do the following:
INFLUXDB_PORT=$(minikube service --url influxdb | cut -d: -f3)
sudo iptables -t nat -A DOCKER ! -i docker0 -p tcp -j DNAT --dport 8086 --to-destination "${MINIKUBE_IP}:${INFLUXDB_PORT}"
You may want to have access to the Kubernetes dashboard with Rancher Desktop:
The following Powershell commands are adapted from https://rancher.com/docs/k3s/latest/en/installation/kube-dashboard/ for a Windows 10 environment with Rancher Desktop.
$Response = Invoke-WebRequest -URI https://github.com/kubernetes/dashboard/releases/latest -MaximumRedirection 0 -ErrorAction Ignore
$Version = $Response.Headers.Location.split("/")[-1]
kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${Version}/aio/deploy/recommended.yaml
Download dashboard.admin-user.yml and dashboard.admin-user-role.yaml the "kube-dashboard" section of the k3s documentation listed above.
kubectl create -f dashboard.admin-user.yml -f dashboard.admin-user-role.yml
kubectl proxy
kubectl -n kubernetes-dashboard describe secret
kubectl -n kubernetes-dashboard describe secret admin-user-token-<k9-65r>
Copy the token value for logging into the dashboard listed below.
The Kubernetes resources are currently stored in separate manifest files. The start.sh and stop.sh shell scripts have been provided to automate the process of applying and deleting the manifest files with kubectl.
NOTE: If Rancher Desktop is set to use Debian but your default WSL2 distro is Ubuntu, you may need to explicitly specify the WSL distro:
wsl -d Debian bash start.sh
Go into the Rancher Desktop app and forward the InfluxDB deployment to a local port. Then execute the following PowerShell commands, replacing the 51785 port number with the port number listed in the GUI for InfluxDB.
$postParams = @{q='CREATE DATABASE footprint_tracking'}
Invoke-WebRequest -Uri http://127.0.0.1:51785/query -Method POST -Body $postParams
See zephyr_footprint_tracking/README.md for details on how to deploy a pod to import the Zephyr Footprint Tracking data into InfluxDB.
See zephyr_test_results/README.md for details on how to deploy a pod to import the Zephyr Nightly Test Results into InfluxDB.
- Grafana: http://localhost:3000/
- Chronograf: http://localhost:8888/
- Kubernetes dashboard: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
- Set up private namespacing?
- Monitor your infrastructure with InfluxDB and Grafana on Kubernetes
- How to spin up the TICK stack in a Kubernetes instance
- Install and Configure Grafana on Kubernetes
- Kubernetes homelab by lisenet
- Minikube Documentation
Here are some useful minikube commands for troubleshooting problems.
$ minikube version
minikube version: v1.25.2
commit: 362d5fdc0a3dbee389b3d3f1034e8023e72bd3a7
Check the status of minikube
$ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
Check minikube addons
$ minikube addons list
Get information about the cluster:
kubectl cluster-info
Kubernetes control plane is running at https://192.168.49.2:8443
CoreDNS is running at https://192.168.49.2:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Check the kubectl and minikube kubectl versions
$ kubectl version -o yaml
$ minikube kubectl -- version -o yaml