- Get from 2 to 5 or more nodes with Ubuntu installed.
- Get the SSH key with root access to the nodes.
git clone https://gitlab.anttek.io/devops/k8s-cluster-kubeadm.git
cd k8s-cluster-kubeadm
python3 -m venv venv
On Linux and macOS:
. venv/bin/activate
On Windows:
.\venv\Scripts\activate
pip install ansible
cp -R example_cluster cluster
Now edit the inventory file and replace the example values with your own.
Ansible will only installs and configures all the necessary packages on the nodes.
Packages installed:
- containerd
- runc
- CNI plugins
- kubeadm
- kubelet
- kubectl
You will still need to initialize the cluster and join the nodes to it manually.
ansible-playbook playbook.yml -b
Please, refer to the official documentation to get more information about the
kubeadm init
command.
In some cases, you may need to add the --apiserver-advertise-address
and --apiserver-cert-extra-sans
flags to the kubeadm init
command.
It depends on your network configuration.
In my case, without these flags, the API server was not accessible from pods.
Example:
kubeadm init --apiserver-advertise-address=0.0.0.0 --apiserver-cert-extra-sans=<CURRENT_NODE_EXTERNAL_IP>,<CURRENT_NODE_INTERNAL_IP>,api-server.<CLUSTER_DOMAIN>
After the cluster is initialized, refer to the
next section of the official documentation.
You may need to join other nodes to the cluster and download the kubeconfig file.
From there you're free to do whatever you want with the cluster.
However, you still need to install a pod network add-on to make the cluster usable.
All the steps below are just my personal preferences.
What I'm going to do:
- Install a pod network add-on: Cilium
- Add persistent storage: Ceph, Kubernetes integration
- Install cert-manager: Cert-manager
- Install an ingress controller: Ingress-nginx
After some searching and comparisons, I decided to use Cilium as the pod network add-on. Installation is pretty straightforward. I recommend you to follow the official documentation.
In my case, there was a network issue when Cilium tried to communicate with the Kubernetes API server. I had to add the
--set k8sServiceHost
and--set k8sServicePort
flags to thecilium install
command.
--k8sServiceHost
is the Kubernetes API server address.
It might be one of the addressess you specified in the--apiserver-cert-extra-sans
flag.
I decided to use Ceph as the persistent storage for the cluster. Because of deploying Ceph cluster is out of the scope of this manual, please refer to this article.
Refer to the official documentation to get more information about the installation process.
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.0/cert-manager.yaml
There is an example of the cert-manager-issuers.yaml
file in the kubernetes
directory.
There are two ClusterIssuers in the file:
letsencrypt-prod
- for productionletsencrypt-staging
- for testing
Update the kubernetes/cert-manager-issuers.yaml
file with your email address and apply it to the cluster.
kubectl apply -n cert-manager -f kubernetes/cert-manager-issuers.yaml
kubectl apply -f kubernetes/ingress-nginx.yaml
NOTE: As I don't have a load balancer, I'm using the
hostNetwork: true
option.
Instead ofDeployment
I'm usingDaemonSet
to run the ingress controller on all the nodes.