/quick-kubernetes-cluster-aws

This repo is a CDK stack that deploys an EKS Kubernetes cluster, a repo for container images, and an autoscaler for nodes.

Primary LanguageTypeScript

What does this do?

This is a CDK stack that quickly gets you setup with a kubernetes cluster, repo, autoscaler on AWS!

What to configure

There are a number of things you can configure in index.ts:

  1. clusterName to change the name of the cluster
  2. kubernetes version: currently CDK supports up to 1.21. Make sure to change both eks.KubernetesVersion.V1_21 and autoscalerImageTag. The image tag is used for the autoscaler-deployment so search through releases here i.e. "1.21" to find the correct tag for your kubernetes version.
  3. the EC2 instance/family. I have it set to graviton3 right now. Due to a quirk with CDK, since I'm using arm64 EC2 instances for my cluster, we have to set both instanceTypes and amiType for everything to work. If you aren't using a graviton/arm64 instance just remove amiType altogether.
  4. for the autoscaler you can of course change minSize and maxSize and probably you should because my current configuration here has a max size of 80!
  5. the maximum retention time for images in the repository. I have it set to 365 day image retention but you can change it to anything you want or simply comment out that line for images to live forever.

Autoscaler

When you deploy an EKS cluster you have to manually deploy the autoscaler-deployment yourself, otherwise your ASG wont add or subtract nodes to your kubernetes cluster when you add or remove pods and other resources. There are a lot of manual steps to do this, so I added them to this CDK stack.

It works for now, but Amazon/Kubernetes update the autoscaler-deployment over time, so I think in the future it might break. If it does, I suggest reading the AWS documentation here and diffing their YAML file converted to JSON with the JSON manifest in this repo to see what to changed.

Alternatively I just learned about Karpenter which might be a better alternative to the autoscaler-deploymment on its own and which AWS created themselves! You could comment out the autoscale stuff in index.ts (step 5) and just set this up instead.

Instructions

  1. edit the variables in index.ts as you want.
  2. in package.json change --profile dev to whichever awscli profile you use with your account, or just remove --profile dev altogether if you dont have multiple accounts setup. I hard coded it to this for safety.
  3. run npm run deploy which will build and deploy the stack!
  4. write down the accessKeyId and grab the secret from secret manager in AWS console add aws cli profile for cluster to your machine: aws configure --profile clusterName KUBECONFIG=$HOME/.ssh/clusterName.kubeconfig aws eks update-kubeconfig --name clusterName --profile clusterName --region us-east-1

for the container registry:

ACCOUNT_ID=<PUT YOUR AWS ACCOUNT ID HERE>
REGION=us-east-1
AWS_CLI_PROFILE=test-runner-repo-user

aws ecr get-login-password --region $REGION --profile $AWS_CLI_PROFILE | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com

And that's it. You should now have a kubernetes cluster that autoscales, along with a repo the cluster can talk to and that you can too to push/pull images!

other CDK commands:

* `npm run build`   compile typescript to js
* `npm run watch`   watch for changes and compile
* `npm run test`    perform the jest unit tests
* `cdk deploy`      deploy this stack to your default AWS account/region
* `cdk diff`        compare deployed stack with current state
* `cdk synth`       emits the synthesized CloudFormation template