Repository for provision Kubernetes(EKS) with following components.
- EKS Cluster
- OpenID integration for provision IAM based authentication and Authorization to EKS cluster
- EFS for shared storage(persistence volume for containers) - RBAC Acces to users
- Bastion host for user operations (one entry point to environment)
- Nat Gateway (govern external routes)
- Worker nodes (used as Spark Executors)
- Autoscaling groups (dynamically scale up and down based of spark execution)
- configure Application namespace (dataengineering)
- Prometheous and pushgateway to monitor and accept app metrics(efs as storage to get multi AZ).
- Grafana for dashborads and integrate with prometheous(efs as storage to get multi AZ).
- Install Argo Scheduler and configure S3 for app cache.
- Provision Cognito and User App integration
- Configure UI authentication for Argo and Grafana from Cognito
You can download source from SCM.
terraform init
./instal_eks.sh terraform init
./uninstal_eks.sh Please feel free to raise a pull request in case if you feel like something can be updated or contributed