Cloudwatch Insight Prometheus Exporter

Cloudwatch Insight Prometheus Exporter allows you to generate Prometheus-compatible metrics using AWS Cloudwatch Log Insights.

Many AWS services create logs that you can see in AWS Cloudwatch Logs, making generating metrics impossible.

To be able to generate metrics, we developed this Prometheus Exporter so we can generate metrics from Cloudwatch Logs using Cloudwatch Log Insights.

Table of contents

Getting started
- Deployment
Configuration

Getting Started

Quick Start - EKS Audit logs

Deployment

Helm chart is located in deploy/soprano directory and you can easily setup the helm chart shown at below.

  $ export BASE_PATH=$(PWD)/deploy/soprano
  
  $ helm upgrade -i aws-cw-prometheus-syncer ${BASE_PATH} -f ${BASE_PATH}/values.yaml

Caveats

To make this tool able to fetch the results of the CloudwatchInsights queries you should make sure the permissions are setup true.

Permissions

{
  "Version": "2012-10-17",
  "Statement": [
      {
        "Sid": "LogGroupRetention",
        "Effect": "Allow",
        "Action": [
            "logs:Get*",
            "logs:Describe*",
            "logs:StartQuery",
            "logs:StopQuery",
            "logs:GetQueryResults",
            "cloudwathc:Get*"
        ]
        "Resource": "arn:aws:logs:REGION:ACCOUNT_ID:log-group:LOG-GROUP-NAME"
      }
  ]
}

This tools is also able to access AWS API via IRSA [Recommended way] assumed way so basically you can setup the IRSA like that;

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: ROLE_ARN

Configuration

You can check the helm configuration details overs there ;

Parameter	Description	Default
`image.repository`	Image	`emirozbir/soprano
`eks.region`	The aws region where you are working	`eu-west-1
`eks.log_group_name`	Log group address to work on	`/aws/eks/test-cluster/cluster
`servicemonitor.exporter_key`	Release name of the kube-prometheus stack	`eu-west-1
`servicemonitor.namespace`	Prometheus operator namespace on	`eu-west-1
`exporter.port`	Exposed port value on	`9877
`serviceAccount.annotations`	Annocations for the IRSA usage or generic purpose staff on	`eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/ROLE-FOR-SOPRANO"

Important: