terraform-kvm-kubespray

Set up HA Kubernetes cluster using KVM, Terraform and Kubespray.

Requirements

Git
Cloud-init
Ansible >= 2.9
Terraform >= 1.0.0
KVM - Kernel Virtual Machine
Libvirt provider >= 0.6.3
Internet connection

Libvirt provider

IMPORTANT:

Bridging VMs with the LAN network only works with libvirt provider 0.6.3 due to the missing implementation of qemu-agent in higher versions. Version 0.6.3 is not included in the Terraform registry and therefore requires manual installation of the provider. (Setup libvirt provider).

Getting Started

If you encounter any issues during the installation, please refer to the troubleshooting page first.

Clone the project and move to the main directory:

git clone https://github.com/MusicDin/terraform-kvm-kubespray.git

cd terraform-kvm-kubespray

SSH keys

Generate SSH keys that will be used to access created VMs:

ssh-keygen

Follow the instructions to generate SSH keys:

Generating public/private rsa key pair.
Enter file in which to save the key (/home/your_username/.ssh/id_rsa): [1]
Enter passphrase (empty for no passphrase): [2]
Enter same passphrase again: [2]
...

[1] You will be asked to enter file in which to save the key. Default is /home/your_username/.ssh/id_rsa.

[2] When asked to enter a password, press ENTER twice to skip setting a password. DO NOT enter it, otherwise Terraform will fail to initialize a cluster.

Finally, you have to enter a location of SSH private key in the vm_ssh_private_key field in terraform.tfvars file.

Cluster setup

Change variables in terraform.tfvars file to fit your needs. Variables are set to work out of the box. Only unset required variable is:

vm_image_source URL or path on the file system to OS image

IMPORTANT: Review variables before initializing a cluster.

Note: Script also supports deployment of single node cluster.

Execute terraform script:

# Initializes terraform project
terraform init

# Shows what is about to be done
terraform plan

# Runs/creates project
terraform apply

Note: The installation process can take up to 20 minutes depending on the configuration.

Test cluster

All configuration files will be generated in config/ directory, and one of them will be admin.conf which is actually a kubeconfig file.

Test if the cluster works by displaying all cluster nodes:

kubectl --kubeconfig=config/admin.conf get nodes

Cluster management

Adding worker nodes to the cluster

In terraform.tfvars file add new worker node(s) in worker_nodes list.

Execute terraform script to add a worker (workers):

terraform apply -var 'action=add_worker'

Removing worker nodes from the cluster

In terraform.tfvars file remove worker node(s) from worker_nodes list.

Execute terraform script to remove a worker (workers):

terraform apply -var 'action=remove_worker'

Upgrading the cluster

In terraform.tfvars file modify:

k8s_kubespray_version and
k8s_version.

Note: Before upgrading make sure Kubespray supports provided Kubernetes version.

Execute terraform script to upgrade a cluster:

terraform apply -var 'action=upgrade'

IMPORTANT: Do not skip releases when upgrading--upgrade by one tag at a time. For more information read Kubespray upgrades.

Destroying the cluster

To destroy the cluster, simply run:

terraform destroy

Related projects

If you are interested in deploying a Kubernetes cluster on vSphere instead of KVM check out this project.

Having issues?

In case you have found a bug, or some unexpected behaviour please open an issue.

If you need anything else, you can contact me on GitHub.

License

Apache License 2.0

sherwinwangs/terraform-kvm-kubespray