/kube-state-metrics

Add-on agent to generate and expose cluster-level metrics.

Primary LanguageGoApache License 2.0Apache-2.0

Overview

kube-state-metrics is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects. (See examples in the Metrics section below.) It is not focused on the health of the individual Kubernetes components, but rather on the health of the various objects inside, such as deployments, nodes and pods.

The metrics are exported through the Prometheus golang client on the HTTP endpoint /metrics on the listening port (default 80). They are served either as plaintext or protobuf depending on the Accept header. They are designed to be consumed either by Prometheus itself or by a scraper that is compatible with scraping a Prometheus client endpoint. You can also open /metrics in a browser to see the raw metrics.

Requires Kubernetes 1.2+

Metrics

There are many more metrics we could report, but this first pass is focused on those that could be used for actionable alerts. Please contribute PR's for additional metrics!

WARNING: THESE METRIC/TAG NAMES ARE UNSTABLE AND MAY CHANGE IN A FUTURE RELEASE.

Metric name Labels/tags
nodes ready=<true|false>
deployment_replicas name=<deployment-name> namespace=<deployment-namespace>
deployment_replicas_available name=<deployment-name> namespace=<deployment-namespace>
container_restarts name=<container-name> namespace=<pod-namespace> pod_name=<pod-name>

kube-state-metrics vs. Heapster

Heapster is a project which fetches metrics (such as CPU and memory utilization) from the Kubernetes API server and nodes and sends them to various time-series backends such as InfluxDB or Google Cloud Monitoring.

While Heapster's focus is on forwarding metrics already generated by Kubernetes, kube-state-metrics is focused on generating completely new metrics from Kubernetes' object state (e.g. metrics based on deployments, replica sets, etc.). The reason not to extend Heapster with kube-state-metrics' abilities is because the concerns are fundamentally different - while Heapster only needs to fetch, format and forward metrics that already exist, kube-state-metrics holds an entire snapshot of Kubernetes state in memory and continuously generates new metrics based off of it but has no responsibility for exporting its metrics anywhere.

In other words, kube-state-metrics itself is designed to be another source for Heapster (although this is not currently the case).

Additionally, some monitoring systems such as Prometheus do not use Heapster for metric collection at all and instead implement their own. Having kube-state-metrics as a separate project enables access to these metrics from those monitoring systems.

Usage

Simply build and run kube-state-metrics inside a Kubernetes pod which has a service account token that has read-only access to the Kubernetes cluster.

Development

When developing, test a metric dump against your local Kubernetes cluster by running:

go run main.go --apiserver=<APISERVER-HERE> --in-cluster=false --port=<APISERVER-PORT> --dry-run