Welcome!
This repository contains a project for visualizing data from Kubecost in Amazon QuickSight, as part of CID (Cloud Intelligence Dashboards).
The dashboard provides visibility into EKS in-cluster cost and usage in a multi-cluster environment, using data from a self-hosted Kubecost pod.
This project can work with any Kubecost tier:
- Kubecost EKS-optimized bundle
- Kubecost free tier
- Kubecost enterprise tier, with the following limitations:
- Data for all clusters are included in a single file per day instead of file per cluster per day
- AWS account ID will not be shown
- The
properties.eksclusternamedataset field will show the primary cluster name for any cluster.
Instead, you can customize the dashboard and use theproperties.clusterfield
Please note that OpenCost is not supported.
More information on the Kubecost EKS-optimized bundle can be found in the below resources:
- Launch blog post
- AMP integration blog post
- Multi-cluster visibility blog post
- Amazon Cognito integration blog post
- EKS user guide
- Kubecost on the AWS Marketplace
- EKS Blueprints Add-on
- EKS Add-on
More information on Kubecost pricing can be found here.
Feature comparison between Kubecost tiers can be found here.
The following is the solution's architecture:
This solution is composed of the following components (in high-level):
- A data collection pod (referred to as "Kubecost S3 Exporter" throughout some parts of the documentation).
It's used to collect the data from Kubecost and upload it to an S3 bucket that you own. - A pipeline that makes the data available to be queried in Athena
- A QuickSight dashboard, along with its QuickSight assets (data source, dataset)
The AWS resources in this solution are deployed using a Terraform module.
The K8s resources in this solution are deployed using a Helm chart.
It's invoked by Terraform by default, but you can choose to deploy it yourself.
The QuickSight dashboard is deployed using the cid-cmd CLI tool.
More detailed information on the architecture and logic are found in the ARCHITECTURE.md file.
Before proceeding to the requirements and deployment of this solution, it's highly recommended you review.
For information related to security in this project, please refer to the SECURITY.md file.
Before proceeding to the deployment of this solution, please complete the requirements, as outlined in the REQUIREMENTS.md file.
For instructions on deploying this solution, please refer to the DEPLOYMENT.md file.
When done, proceed to the Post-Deployment Steps section below.
Before proceeding to use the dashboard, please complete the post-deployment steps outlined in the POST_DEPLOYMENT.md file.
In addition, if you'd like to deploy the Kubecost-S3-Exporter on additional clusters:
See the "Deploying on Additional Clusters" section in the MAINTENANCE.md file.
If you'd like to add/remove K8s labels/annotations to/from the dataset:
See the "Adding/Removing Labels/Annotations to/from the Dataset" section in the MAINTENANCE.md file.
For update instructions, please refer to the UPDATE.md file.
For instructions on common maintenance tasks related to this solution, please refer to the MAINTENANCE.md file.
For instructions on troubleshooting common issues related to this solution, please refer to the TROUBLESHOOTING.md file.
For instructions on cleanup, please refer to the CLEANUP.md file.
See CONTRIBUTING for more information on how to report security issues.
See SECURITY.md for more information related to security in this solution.
This project is licensed under the Apache-2.0 License.