Prometheus Exporter for Google Cloud Platform (GCP)
I want to be able to monitor my resource consumption across multiple cloud platforms (GCP, Digital Ocean and Linode). I was inspired by @metalmatze's DigitalOcean Exporter and, with this exporter, have the three that I need:
Result:
And:
The application uses Google's Application Default Credentials (ADCs) to simplify authentication by finding credentials automatically.
On a machine running gcloud
that's authenticated with your user (e.g. Gmail) account, you can run gcloud auth application-default login
to establish your user account as ADCs. This ensures that the Exporter is able to operate as if it were you(r user account), enumerate GCP projects that you(r user account) has access to and resources within those projects.
If you run the Exporter remotely, you will need to create a service account for it to use. The Exporter will only be able to enumerate projects and project resources that this service account is able to access.
In the following examples, the Exporter's container is configured to use the ADCS stored in ${HOME}/.config/gcloud/appl...
In this example, ADCs will be automatically detected without further configuration.
go get github.com/DazWilkin/gcp-exporter
go run github.com/DazWilkin/gcp-exporter
PORT=9402
CREDENTIALS="${HOME}/.config/gcloud/application_default_credentials.json"
REPO="ghcr.io/dazwilkin/gcp-exporter"
docker run \
--interactive --tty \
--publish=${PORT}:${PORT} \
--volume=${CREDENTIALS}:/secrets/client_secrets.json \
--env=GOOGLE_APPLICATION_CREDENTIALS=/secrets/client_secrets.json \
ghcr.io/dazwilkin/gcp-exporter:437daa490fd8573db462fa117033da237f6cdf65
gcp-exporter
container images are being signed by Sigstore and may be verified:
cosign verify \
--key=./cosign.pub \
ghcr.io/dazwilkin/gcp-exporter:437daa490fd8573db462fa117033da237f6cdf65
NOTE
cosign.pub
may be downloaded here
To install cosign
, e.g.:
go install github.com/sigstore/cosign/cmd/cosign@latest
docker-compose up
NB docker-compose.yml
configuration for gcp-exporter
services is:
gcp-exporter:
image: ghcr.io/dazwilkin/gcp-exporter:437daa490fd8573db462fa117033da237f6cdf65
container_name: gcp-exporter
environment:
- GOOGLE_APPLICATION_CREDENTIALS=/secrets/client_secrets.json
volumes:
- /home/dazwilkin/.config/gcloud/application_default_credentials.json:/secrets/client_secrets.json
expose:
- "9402" # GCP Exporter port registered on Prometheus Wiki
ports:
- 9402:9402
The Docker Compose configuration includes:
NB You will need to create an alertmanager.yml
configuration file. This example shows you how to configure AlertManager to send alerts to Gmail
Assuming MicroK8s and Prometheus Operator
NAMESPACE="gcp-exporter"
kubectl create namespace ${NAMESPACE}
kubectl create secret generic gcp-exporter \
--from-file=client_secrets.json=/home/dazwilkin/.config/gcloud/application_default_credentials.json \
--namespace=${NAMESPACE}
kubectl apply \
--filename=./kubernetes.yaml \
--namespace=${NAMESPACE}
# NB This must be installed to 'monitoring' namespace
kubectl apply --filename=./kubernetes.rule.yaml --namespace=monitoring
Learning about multi-arch builds to run on Raspberry Pi 4.
Unsure how to use docker manifest
with GitHub Actions as this model has been suplanted by docker buildx
(that I don't want to use).
Refactored Dockerfile
to take a build argument GOLANG_OPTIONS
(default=CGO_ENABLED=0 GOOS=linux GOARCH=amd64
)
docker build \
--build-arg=GOLANG_OPTIONS="CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7" \
--tag=ghcr.io/dazwilkin/gcp-exporter:arm32v7 \
--file=./Dockerfile \
.
NOTE See environment variables
git clone git@github.com:DazWilkin/gcp-exporter.git && cd gcp-exporter
Please file issues
Name | Type | Description |
---|---|---|
gcp_exporter_buildinfo |
Counter | A metric with a constant '1' value labeled by OS version, Go version, and the Git commit of the exporter |
gcp_exporter_startime |
Gauge | Exporter start time in Unix epoch seconds |
gcp_compute_engine_instances |
Gauge | Number of instances |
gcp_compute_engine_forwardingrules |
Gauge | Number of forwardingrules |
gcp_kubernetes_engine_cluster_up |
Gauge | 1 if the cluster is running, 0 otherwise |
gcp_kubernetes_engine_cluster_nodes |
Gauge | Number of nodes currently in the cluster |
gcp_storage_buckets |
Gauge | Number of buckets |
Registered 9402
with Prometheus Exporters' default port allocations
Using Google's (now legacy) API Client Libraries. The current Cloud Client Libraries do not provide coverage for all the relevant resources.
- Google Compute Engine API
- Google Resource Manager API && GoDoc
- Google Kubernetes Engine (Container) API && GoDoc