/crdb-cluster-eks-terraform

Primary LanguageHCLMIT No AttributionMIT-0

Deploy CRDB Cluster In EKS Terraform

CockroachDB is a distributed SQL database that aims to make it simple to build, deploy, and operate globally-scaled applications. It is designed to be highly available, survive data center outages, and maintain consistent performance across a geographically distributed database cluster.

One of the key features of CockroachDB is its ability to automatically shard data across multiple nodes in a cluster, allowing it to scale horizontally as more nodes are added.

Table of Contents

  1. About this Repo
  2. Requirements
  3. Target architecture
  4. How to install
  5. References
  6. Related Resources

About this Repo

This repo provides a Terraform Module that can be used to deploy CRDB multi node database cluster in a single EKS cluster using CockroachDB Kubernetes Operator. It his also uses cert-manager to secure node communication with TLS certificate.

Requirements

  • An active AWS account with required permission to deploy resources in Kubernetes cluster
  • A running EKS cluster with version v1.23+ , EKS Node should have kubernets label node=cockroachdb
  • EBS CSI Driver v1.19.0+ installed in the EKS cluster
  • An Understanding of Terraform and usage.
  • Terraform CLI with version 1.0.0+
  • Install Kubectl
  • AWS CLI v2.9.18+
  • A web browser that is supported for use with the AWS Management Console. (See the list of supported browsers)

Target architecture

Image

How to install

Setup

Install

  • terraform init
  • terraform apply

Verification

  • Verify if kubectl is installed. Follow instructions to install kubectl
kubectl version
  • Set the eks context using aws cli
aws eks update-kubeconfig --name <eks_cluster_name> --region <AWS_Region_Name>

  • Verify number of CRDB pods using

    kubectl get pods -n <namespace>

NAME                                  READY   STATUS    RESTARTS   AGE
cockroach-operator-655fbf7847-zn9v8   1/1     Running   0          30m
cockroachdb-0                         1/1     Running   0          24m
cockroachdb-1                         1/1     Running   0          24m
cockroachdb-2                         1/1     Running   0          24m
  • 3 cockroach dB pods will be running as the default value in the variable.tf is set to 3. This may vary based on the value set on variable.tf file.

Scale up/down:

  • Increase/decrease the number of nodes for managed node group in ./main/variable.tf and perform terraform apply

Note - Due to POD anti-affinity rule, only one CRDB Pod can run in one EKS worker note. Hence,use EKS autoscaler or Karpenter for autoscaling worker nodes automcatically to run the additional CRDB nodes (POD)

  • For example, if number of nodes is increased from 3 to 4, you would see 4 pods are running after executing terraform apply

kubectl get pods -n <namespace>

NAME                                  READY   STATUS    RESTARTS   AGE
cockroach-operator-655fbf7847-zn9v8   1/1     Running   0          30m
cockroachdb-0                         1/1     Running   0          24m
cockroachdb-1                         1/1     Running   0          24m
cockroachdb-2                         1/1     Running   0          24m
cockroachdb-3                         1/1     Running   0          30s

Clean Up

To tear down, run

terraform destroy

References

Providers

Name Version
aws 4.66.0

Modules

Name Source Version
common-tags ../modules/common-tags n/a
crdb ../modules/crdb n/a

Resources

Name Type
aws_region.current data source

Inputs

Name Description Type Default Required
eks_cluster_name Name of the EKS Cluster string "demo" no
number_of_nodes Number of nodes to be deployed number 3 no
region set the region for deployment string "us-east-2" no

Outputs

No outputs.

Security

See CONTRIBUTING for more information.

License

Related Resources