- Terraform/Terragrunt
- Contributing
- Requirements
- Quickstart
- Main purposes
- What you get
- Curated Features
- Bottlerocket support
- AWS Session Manager by default
- From and to Zero scaling with EKS Managed Node Groups
- Automatic dependencies upgrade
- Enforced security
- Out of the box logging
- Out of the box monitoring
- Long term storage with Thanos
- Support for ARM instances
- Helm v3 provider
- Other and not limited to
- Always up to date
- Requirements
- Examples
- Additional infrastructure blocks
- Branches
- License
tEKS is a set of Terraform / Terragrunt modules designed to get you everything you need to run a production EKS cluster on AWS. It ships with sensible defaults, and add a lot of common addons with their configurations that work out of the box.
This is our opinionated view of what a well structred infrastructure as code repository should look like.
- Terragrunt implementation is available in the
terragruntfolder.
Contribution are welcome, as well as issues, we are usually quite reactive. If you need more support for your project, do not hesitate to reach us directly.
Quickstart guide is available here or on the official documentation website
The main goal of this project is to glue together commonly used tooling with Kubernetes/EKS and to get from an AWS Account to a production cluster with everything you need without any manual configuration.
A production cluster all defined in IaaC with Terraform/Terragrunt:
- AWS VPC if needed based on
terraform-aws-vpc - EKS cluster base on
terraform-aws-eks - Kubernetes addons based on
terraform-kubernetes-addons: provides various addons that are often used on Kubernetes and specifically on EKS. This module is currated by Particule and well maintained.
Everything is tied together with Terragrunt and allows you to deploy a multi cluster architecture in a matter of minutes.
The additional features are provided by tEKS here as well as our curated addons module which support a bunch of various configuration.
Bottlerocket OS is available for node groups (see example here). Bottle rocket is a container centric OS with less attack surface and no default shell.
All the instances (Bottlerocket or Amazon Linux) are registered with AWS Session Manager. No SSH keys or SSH access is open on instances. Shell access on every instance can be given with SSM for added security.
aws ssm start-session --target INSTANCE_ID
tEKS support scaling to and from 0, even with using well know Kubernetes labels, there are a number of ongoing issues for support of EKS Managed node groups with Cluster Autoscaler. Thanks to automatic ASG tagging, tEKS adds the necessary tags on autoscaling group to balance similar node groups and allow you to scale to and from 0 and even to use well know labels such as node.kubernetes.io/instance-type or topology.kubernetes.io/zone . The logic can be extended to support other well known labels.
We are using renovate to automatically open PR with the latest dependencies update (Terraform modules upgrade) so you never miss an upgrade and are alwasy up to date with the latest features.
- Encryption by default for root volume on instances with Custom KMS Key
- AWS EBS CSI volumes encrypted by default with Custom KMS Key
- No IAM credentials on instances, everything is enforced with IRSA.
- Each addons is deployed in it's own namespace with sensible default network policies.
- Calico Tigera Operator for network policy.
- PSP are enabled but not enforced because of depreciation.
Three stacks are supported:
- AWS for Fluent Bit: Forward containers logs to Cloudwatch Logs
- Grafana Loki: Uses Promtail to forward logs to Loki. Grafana or a tEKS supported monitoring stack (see below) is necessary to display logs.
- Prometheus Operator with defaults dashboards
- Addons that support metrics are enable along with their
serviceMonitor - Custom grafana dashboard are available by default
Two stacks are supported:
- Victoria Metrics Stack: Victoria Metrics is a Prometheus alertnative, compatible with prometheus CRDs
- Kube Prometheus Stack: Classic Prometheus Monitoring
With Prometheus, tEKS includes Thanos by default. Thanos uses S3 to store and query metrics, offering long term storage without the costs. For more information check out our article on the CNCF Blog
With either Amazon Linux or BottleRocket, you can use a mix of ARM and AMD64 instances. Check out our example
- All addons support Helm v3 configuration
- All charts are easily customizable
- priorityClasses for addons and critical addons
- lot of manual stuff have been automated under the hood
We always support the latest modules and features for our addons module.
Our cutting edges addons include (not limited to):
- AWS EBS CSI Drivers: Support for Volume encryption by default, snapshot, etc
- AWS EFS CSI Drivers: Use AWS NFS shares.
- Secret Store CSI Driver: load
secret from Secret Managers with
aws-secret-store-csi-driverdriver - Linkerd2 or Certificate Manager CSI for mTLS
Terragrunt is not a hard requirement but all the modules are tested with Terragrunt.
This repository use pre-commit hooks, please see this on how to setup tooling
ASDF is a package manager which is great for managing cloud native tooling. More info here (eg. French).
for p in $(cut -d " " .tool-versions -f1); do asdf plugin add $p; done
asdf install
terragrunt/live folder provides an opinionated directory structure for a production environment.
If you wish to extend your infrastructure you can pick up additional modules on the particuleio github page. Some modules can also be found on the clusterfrak-dynamics github page.
main: Backward incompatible with v1.X but compatible with v2.X, releases bumped to v3.X because a lot has changed.release-1.X: Compatible with Terraform < 0.12 and Terragrunt < 0.19. Be sure to target the same modules version.release-2.X: Compatible with Terraform >= 0.12 and Terragrunt >= 0.19. Be sure to target the same modules version.
