prometheus-to-cloudwatch

Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch

Usage

NOTE: The module accepts parameters as command-line arguments or as ENV variables (or any combination of command-line arguments and ENV vars). Command-line arguments take precedence over ENV vars

Command-line argument	ENV var	Description
aws_access_key_id	AWS_ACCESS_KEY_ID	AWS access key Id with permissions to publish CloudWatch metrics
aws_secret_access_key	AWS_SECRET_ACCESS_KEY	AWS secret access key with permissions to publish CloudWatch metrics
cloudwatch_namespace	CLOUDWATCH_NAMESPACE	CloudWatch Namespace
cloudwatch_region	CLOUDWATCH_REGION	CloudWatch AWS Region
cloudwatch_publish_timeout	CLOUDWATCH_PUBLISH_TIMEOUT	CloudWatch publish timeout in seconds
prometheus_scrape_interval	PROMETHEUS_SCRAPE_INTERVAL	Prometheus scrape interval in seconds
prometheus_scrape_url	PROMETHEUS_SCRAPE_URL	The URL to scrape Prometheus metrics from
cert_path	CERT_PATH	Path to SSL Certificate file (when using SSL for `prometheus_scrape_url`)
keyPath	KEY_PATH	Path to Key file (when using SSL for `prometheus_scrape_url`)
accept_invalid_cert	ACCEPT_INVALID_CERT	Accept any certificate during TLS handshake. Insecure, use only for testing
additional_dimension	ADDITIONAL_DIMENSION	Additional dimension specified by NAME=VALUE

NOTE: If AWS credentials are not provided in the command-line arguments (aws_access_key_id and aws_secret_access_key) or ENV variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY), the chain of credential providers will search for credentials in the shared credential file and EC2 Instance Roles. This is useful when deploying the module in AWS on Kubernetes with kube2iam, which will provide IAM credentials to containers running inside a Kubernetes cluster, allowing the module to assume an IAM Role with permissions to publish metrics to CloudWatch.

Examples

Build Go program

go get

CGO_ENABLED=0 go build -v -o "./dist/bin/prometheus-to-cloudwatch" *.go

Run locally

export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
export CLOUDWATCH_NAMESPACE=kube-state-metrics
export CLOUDWATCH_REGION=us-east-1
export CLOUDWATCH_PUBLISH_TIMEOUT=5
export PROMETHEUS_SCRAPE_INTERVAL=30
export PROMETHEUS_SCRAPE_URL=http://xxxxxxxxxxxx:8080/metrics
export CERT_PATH=""
export KEY_PATH=""
export ACCEPT_INVALID_CERT=true

./dist/bin/prometheus-to-cloudwatch

Build Docker image

NOTE: it will download all Go dependencies and then build the program inside the container (see Dockerfile)

docker build --tag prometheus-to-cloudwatch  --no-cache=true .

Run in a Docker container

docker run -i --rm \
        -e AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXXXX \
        -e AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
        -e CLOUDWATCH_NAMESPACE=kube-state-metrics \
        -e CLOUDWATCH_REGION=us-east-1 \
        -e CLOUDWATCH_PUBLISH_TIMEOUT=5 \
        -e PROMETHEUS_SCRAPE_INTERVAL=30 \
        -e PROMETHEUS_SCRAPE_URL=http://xxxxxxxxxxxx:8080/metrics \
        -e CERT_PATH="" \
        -e KEY_PATH="" \
        -e ACCEPT_INVALID_CERT=true \
        prometheus-to-cloudwatch

Run on Kubernetes

To run on Kubernetes, we will deploy two Helm charts

kube-state-metrics - to generates metrics about the state of various objects inside the cluster, such as deployments, nodes and pods
prometheus-to-cloudwatch - to scrape metrics from kube-state-metrics and publish them to CloudWatch

Install kube-state-metrics chart

helm install stable/kube-state-metrics

Find the running services

kubectl get services

Copy the name of the kube-state-metrics service (gauche-turtle-kube-state-metrics) into the ENV var PROMETHEUS_SCRAPE_URL in values.yaml. It should look like this:

PROMETHEUS_SCRAPE_URL: "http://gauche-turtle-kube-state-metrics:8080/metrics"

Deploy prometheus-to-cloudwatch chart

cd chart
helm install .

prometheus-to-cloudwatch will start scraping the /metrics endpoint of the kube-state-metrics service and send the Prometheus metrics to CloudWatch

Fork the repo on GitHub
Clone the project to your own machine
Commit changes to your own branch
Push your work back up to your fork
Submit a Pull request so that we can review your changes

NOTE: Be sure to merge the latest from "upstream" before making a pull request!

License

See LICENSE for full details.

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.