`kubernetes-join-command` not delegated to any host except master
iLem0n opened this issue · 6 comments
Trying to bring up a simple k8s cluster with one master and one worker node.
Just bringing them up using vagrant brings me to the following problem:
It seems that the kubernetes-join-command is only be set on the master node not the worker ones.
Which results in failure ad worker provisioning.
Versions:
# VAGRANT:
Vagrant 2.3.4
# ANSIBLE:
ansible [core 2.14.1]
config file = /Users/ilem0n/projects_NEW/multiserver-cluster-setup/node-config/ansible.cfg
configured module search path = ['/Users/ilem0n/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.10/site-packages/ansible
ansible collection location = /Users/ilem0n/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.10.10 (main, Feb 8 2023, 05:40:53) [Clang 14.0.0 (clang-1400.0.29.202)] (/usr/local/opt/python@3.10/bin/python3.10)
jinja version = 3.1.2
libyaml = True
Vagrant file:
IMAGE_NAME = "debian/bullseye64"
N = 1
SYSTEM_USER = "vagrant"
Vagrant.configure("2") do |config|
config.ssh.insert_key = false
config.vm.provider "virtualbox" do |node|
node.memory = 2056
node.cpus = 2
end
config.vm.define "kubernetes-master" do |master|
master.vm.box = IMAGE_NAME
master.vm.network "private_network", ip: "192.168.0.10"
master.vm.network "forwarded_port", guest: 80, host: 80
master.vm.network "forwarded_port", guest: 443, host: 443
master.vm.network "forwarded_port", guest: 6443, host: 6443
master.vm.hostname = "kubernetes-master"
master.vm.provision "ansible" do |ansible|
ansible.verbose = "vvv"
ansible.playbook = "master-playbook.yml"
ansible.extra_vars = {
role: "control_plane",
node_ip: "192.168.0.10"
}
end
end
(1..N).each do |i|
config.vm.define "kubernetes-worker-#{"%02d" % i}" do |node|
node.vm.box = IMAGE_NAME
node.vm.network "private_network", ip: "192.168.56.#{i + 10}"
node.vm.hostname = "kubernetes-worker-#{"%02d" % i}"
node.vm.provision "ansible" do |ansible|
ansible.verbose = "vvv"
ansible.playbook = "master-playbook.yml"
ansible.extra_vars = {
role: "node",
node_ip: "192.168.56.#{i + 10}"
}
end
end
end
endmaster-playbook.yml
---
- hosts: all
name: Kubernetes configuration
become: true
gather_facts: true
vars:
system_user: ilem0n
collections:
- geerlingguy.k8s
- kubernetes.core
pre_tasks:
- name: Preparation
ansible.builtin.include_tasks: sub-tasks/pre-tasks.yml
roles:
- helm
- andrewrothstein.k9s
- geerlingguy.containerd
- role: geerlingguy.ntp
ntp_timezone: Europe/Berlin
- role: geerlingguy.kubernetes
kubernetes_role: "{{ role }}"
kubernetes_config_init_configuration:
localAPIEndpoint:
advertiseAddress: "{{ kubernetes_apiserver_advertise_address | default(ansible_default_ipv4.address, true) }}"join-command setup:
...
TASK [geerlingguy.kubernetes : Set the kubeadm join command globally.] *********
task path: /Users/ilem0n/.ansible/roles/geerlingguy.kubernetes/tasks/main.yml:48
ok: [kubernetes-master] => (item=kubernetes-master) => {
"ansible_facts": {
"kubernetes_join_command": "kubeadm join 10.0.2.15:6443 --token dbdld5.4djx6olxur7zswo7 --discovery-token-ca-cert-hash sha256:2d2a27846899fa8f9b731bf416147edd806e1d8b27a93ff7513085e9f893508d \n"
},
"ansible_loop_var": "item",
"changed": false,
"item": "kubernetes-master"
}
...
nodes tries to use join-command:
...
TASK [geerlingguy.kubernetes : Join node to Kubernetes control plane.] *********
task path: /Users/ilem0n/.ansible/roles/geerlingguy.kubernetes/tasks/node-setup.yml:2
fatal: [kubernetes-worker-01]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: 'kubernetes_join_command' is undefined. 'kubernetes_join_command' is undefined\n\nThe error appears to be in '/Users/ilem0n/.ansible/roles/geerlingguy.kubernetes/tasks/node-setup.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Join node to Kubernetes control plane.\n ^ here\n"
}
...
This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!
Please read this blog post to see the reasons why I mark issues as stale.
I seem to have run into this issue as well. Is there a band-aid/temp solution or workaround for this?
I seem to have run into this issue as well. Is there a band-aid/temp solution or workaround for this?
I think I've solved my issues for now, as a temp fix. Leaving this info here for the next person.
A bit more context to my situation: I'm trying to get geerlingguy's raspberry-pi-dramble to work. Even though it's archived etc etc.
I've changed my version of kubernetes in main.yml from 1.19.70 to 1.25.1-00
I ran sudo kubeadm init on kube1. Which gave me a a bit of additional troubleshooting I couldn't get from doing -vvvvv in the playbook.
That told me to fix 2 settings. Both errors I googled and I found the following two commands I could run:
$ sudo sysctl -w net.ipv4.ip_forward=1$ sudo modprobe br_netfilter
After doing this it completed and spat out a $ kubeadm join [ip address]:6443 --token [token] --discovery-token-ca-cert-hash [sha256]
This I could use on the other Kubes (2, 3 and 4)
I had to run these 3 commands on the other kubes, which I simplified by doing:
$ sudo sysctl -w net.ipv4.ip_forward=1 && sudo modprobe br_netfilter$ sudo kubeadm join [ip address]:6443 --token [token] --discovery-token-ca-cert-hash [sha256]
They all neatly joined kube1.
To make sure I did not get stuck running the playbook, I chose for the quick and dirty 'remove from playbook'.
$ nano /home/user/.ansible/roles/geerlingguy.kubernetes/tasks/node-setup.yml
and commenting out the 'Join node to Kubernetes control plane.'
Those sysctl commands should run within this playbook. If not, please comment to #146
To run the node-setup successfully, it is neccessary to run the control-plane AND the node-setup in one run, as the kubernetes-join-command needs the control-plane to get the command.
(It is no problem to run the control-plane-setup multiple times, e.g. to add another worker-node)
So I am not sure how to do this in vagrant, as the node-setup depends on the control-plane-setup. This is because this is done with kubeadm-commands. The token to join is not saved in a file, but is read from control-plane during the run of the playbook. @iLem0n
This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!
Please read this blog post to see the reasons why I mark issues as stale.
This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.