This tool simply watches Kubernetes Events and logs them to stdout in JSON to be collected and stored by your logging solution, e.g. fluentd, fluent-bit, Filebeat, or Promtail. Other tools exist for persisting Kubernetes Events, such as Sysdig, Datadog, or Google's event-exporter but this tool is open and will work with any logging solution.
Events in Kubernetes log very important information. If are trying to understand what happened in the past then these events show clearly what your Kubernetes cluster was thinking and doing. Some examples:
- Pod events like failed probes, crashes, scheduling related information like
TriggeredScaleUp
orFailedScheduling
- HorizontalPodAutoscaler events like scaling up and down
- Deployment events like scaling in and out of ReplicaSets
- Ingress events like create and update
The problem is that these events are simply API objects in Kubernetes and are only stored for about 1 hour. Without some way of storing these events, debugging a problem in the past very tricky.
Example of events:
39m Normal UpdatedLoadBalancer Service Updated load balancer with new hosts
40m Normal SuccessfulDelete DaemonSet Deleted pod: ingress02-nginx-ingress-controller-vqqjp
41m Normal ScaleDown Node node removed by cluster autoscaler
54m Normal Started Pod Started container
55m Normal Starting Node Starting kubelet.
55m Normal Starting Node Starting kube-proxy.
55m Normal NodeAllocatableEnforced Node Updated Node Allocatable limit across pods
55m Normal NodeReady Node Node ip-10-0-23-14.compute.internal status is now: NodeReady
58m Normal SuccessfulCreate DaemonSet Created pod: ingress02-nginx-ingress-controller-bz7xj
58m Normal CREATE ConfigMap ConfigMap default/ingress02-nginx-ingress-controller
Use the Helm chart from this repo:
helm install chart/
Or use the chart from deliveryhero/helm-charts/stable/k8s-event-logger:
helm repo add deliveryhero https://charts.deliveryhero.io/
helm install deliveryhero/k8s-event-logger
Or use helm chart from github pages:
helm repo add k8s-event-logger https://max-rocket-internet.github.io/k8s-event-logger
helm install k8s-event-logger/k8s-event-logger
Or use the pre-built image maxrocketinternet/k8s-event-logger
If you're unable to use the pre-built image, you can build it yourself:
make all IMG=<your-container-registry>/k8s-event-logger TAG=latest
This uses docker buildx
to create a multi-platform image. To set up your build host system to be able to build these images, see this guide or make all
and review the Makefile for what it does.
Or to just build locally for testing without multi-arch support (but also doesn't require a registry):
docker build --tag localhost/k8s-event-logger .
Run it:
go run main.go