/k8s-terraform

Primary LanguageHCL

Terraform template to create k8s cluster on GCP

This repository contains terraform code and associated scripts to create k8s cluster on GCP. It creates following resources:-

  • VPC Network
  • Associated Subnet
  • Firewall
  • Master Node
  • "N" number of worker nodes
  • "gvisor" based worked node
  • Create any cluster version supported on Ubuntu18 (OS is still hardcoded)

Following variables can be passed

  • GCP Project name
  • Cloud region for cluster creation
  • Service and POD CIDR's
  • Network plugin - calico and weavenet
  • kubernetes version

Pre-requisite

I have tested it on Ubuntu18 desktop, if anybody tests it on windows let me know. I will update Have a GCP account and log in to it. Whichever account below comand is using will be used by the terraform (projects can be different)

gcloud compute instances list

Usage

Create cluster without gvisor

terraform init
terraform apply -var="gcp_project=" -auto-approve

Then login to master and wait for install to finish (takes about 1-2 mins). Last of the the log will state deployment completed.

gcloud compute ssh k8s-master
cat /var/log/kubeadm-init.log

Copy kubeadm join command from the log and execute it on worker nodes (and gvisor node if created) On worker node (remeber sudo)

sudo kubeadm join 10.1.0.4:6443 --token xxxxxxxxxxxxxxxxxx \
    --discovery-token-ca-cert-hash yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

Customize deployment with variables

You can choose the flags and values based on definition in "variables.tf" file.

terraform init
terraform   apply  -var="gvisor=y" -var="num_of_workers=2" -var="cni_provider=calico" -auto-approve

Creating cluster with older version for upgrade task

terraform apply -var="k8s_version=1.18.3" -auto-approve

Providing inputs via file

You can create file terraform.tfvars, place it in root directory, and provide values in it, like below

gcp_project="yourproject"
gvisor="y"
num_of_workers=2
cni_provider="calico"

you can then run terraform command as below

terraform apply -auto-approve

Deleting the resources

Instead of apply use destroy command like below

terraform destroy  -auto-approve
terraform destroy  -var="gvisor=y" -var="num_of_workers=2" -var="cni_provider=calico" -auto-approve

Advanced Usage

Adding nodes after initial creation

Below command will add 2 extra nodes after you have created initial cluster

terraform apply -var="num_of_workers=3" -auto-approve

If below command is executed after the above command, it will take out 2 nodes from cluster (make sure you know what you are doing)

terraform apply -var="num_of_workers=1" -auto-approve

Adding gvisor and removing it

Add gvisor node to existing cluster

terraform apply -var="gvisor=y" -auto-approve

Remove gvisor node from existing cluster (Do drain and delete node)

terraform apply -var="gvisor=N" -auto-approve