innobead/kubefire

kubefire install on Debian GNU/Linux 11 (bullseye) => expected: 0.4.0/kubefire-cni-bridge

mjbright opened this issue · 3 comments

mjbright commented

Describe the bug
I installed kubefire v0.3.8 on Debian Bullseye (specifically from a proxmox-ve v7.1-1 iso) - no errors at install.

But kubefire info tells me "expected: 0.4.0/kubefire-cni-bridge"

Creating a cluster using kubefire cluster create demo seems to go well (kubeadm init seems successful) but fails finally as shown below:

INFO[2023-04-14T17:38:51+02:00] 10.62.0.3:22:
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.62.0.3:6443 --token 75sj9p.o2uw58h3vredhdft \
	--discovery-token-ca-cert-hash sha256:9234ce0490866b7e066cbddf23bfcca30c1c3d5c060ad9e42d2ac29d905d53d7
INFO[2023-04-14T17:38:51+02:00] running kubeadm token create --print-join-command  node=demo-master-1
INFO[2023-04-14T17:38:51+02:00] creating the join command
INFO[2023-04-14T17:38:51+02:00] running KUBECONFIG=/etc/kubernetes/admin.conf kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.9.6/install/kubernetes/quick-install.yaml  node=demo-master-1
INFO[2023-04-14T17:38:51+02:00] applying CNI network
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: serviceaccount/cilium created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: serviceaccount/cilium-operator created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: configmap/cilium-config created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: clusterrole.rbac.authorization.k8s.io/cilium created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: clusterrole.rbac.authorization.k8s.io/cilium-operator created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: clusterrolebinding.rbac.authorization.k8s.io/cilium created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: clusterrolebinding.rbac.authorization.k8s.io/cilium-operator created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: Warning: spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the "priorityClassName" field instead
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: daemonset.apps/cilium created
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: deployment.apps/cilium-operator created
INFO[2023-04-14T17:38:52+02:00] running KUBECONFIG=/etc/kubernetes/admin.conf kubectl taint nodes --all node-role.kubernetes.io/master-  node=demo-master-1
INFO[2023-04-14T17:38:52+02:00] untainting the master node                    node=demo-master-1
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: error: taint "node-role.kubernetes.io/master" not found
Usage:
  kubefire cluster create [name] [flags]

Flags:
  -b, --bootstrapper string    Bootstrapper type, options: [kubeadm, k3s, rke2, k0s] (default "kubeadm")
  -c, --config string          Cluster configuration file (ex: use 'config-template' command to generate the default cluster config)
  -o, --extra-options string   Extra options (ex: key=value,...) for bootstrapper
  -f, --force                  Force to recreate if the cluster exists
  -h, --help                   help for create
  -i, --image string           Rootfs container image (default "ghcr.io/innobead/kubefire-opensuse-leap:15.2")
      --kernel-args string     Kernel arguments (default "console=ttyS0 reboot=k panic=1 pci=off ip=dhcp security=apparmor apparmor=1")
      --kernel-image string    Kernel container image (default "ghcr.io/innobead/kubefire-ignite-kernel:4.19.125-amd64")
      --master-count int       Count of master node (default 1)
      --master-cpu int         CPUs of master node (default 2)
      --master-memory string   Memory of master node (default "2GB")
      --master-size string     Disk size of master node (default "10GB")
      --no-cache               Forget caches
      --no-start               Don't start nodes
  -k, --pubkey string          Public key
  -v, --version string         Version of Kubernetes supported by bootstrapper (ex: v1.18, v1.18.8, empty)
      --worker-count int       Count of worker node
      --worker-cpu int         CPUs of worker node (default 2)
      --worker-memory string   Memory of worker node (default "2GB")
      --worker-size string     Disk size of worker node (default "10GB")

Global Flags:
  -t, --github-token string   GIthub Personal Access Token used to query repo release info
  -l, --log-level string      log level, options: [panic, fatal, error, warning, info, debug, trace] (default "info")

FATA[2023-04-14T17:38:52+02:00] failed to run kubefire                        error="failed to deploy cluster (demo): Process exited with status 1"

This does succeed to perform a single-node installation.

Connecting to the demo-master-1 node we can validate that the kubeadm init was successful

KUBECONFIG=/etc/kubernetes/admin.conf kubectl get no
NAME            STATUS   ROLES           AGE     VERSION
demo-master-1   Ready    control-plane   2m41s   v1.27.0

To Reproduce
Steps to reproduce the behavior with debug log enabled. --log-level=debug

  • run the kubefire info command to see that there is a kubefire-cni-plugin related problem
  • create a demo cluster using kubefire cluster create demo

Expected behavior

  • kubefire info should show that kubefire-cni-plugin is present (version 0.4.0)
  • demo cluster creation should succeed in creating a 2-node cluster

Environment

  • OS version
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
  • Kubefire version
Build: 46925d5-20220621143922```

 - Kubefire runtime info
```kubefire info
CONTAINERD	IGNITE 	CNI   	RUNC 	CNIPLUGIN
v1.6.6    	v0.10.0	v1.1.1	1.1.3	 (expected: 0.4.0/kubefire-cni-bridge)

Additional context
The OS was installed from a Proxmox-VE iso image. I suspect that some debian package required by kubefire is missing.

mjbright commented

In fact, this is working better than I initially thought.

After creating a 2-node k0s-cluster I realize that cluster creation works fine and I can setup routes to Pod and Cluster subnetworks and access a Service, or Pods fine.

With regard to the initial issue when runningkubefire cluster create demo

  • kubefire info does say expected: 0.4.0/kubefire-cni-bridge
  • The cluster create finished with a failure, but maybe the single-node cluster was OK.

I'd expected kubefire cluster create demo to create a 2-node cluster, and I was badly interpreting the kubefire cluster show demo output

So there is something not right, but this is not blocking.

I am able to successfully create a 2-node k0s-cluster on this Debian (Proxmox-VE) machine.

  • 👍1
innobead commented

This is the same cause as #75 . Thanks for reporting this and this will be fixed in the next version.

INFO[2023-04-14T17:38:52+02:00] untainting the master node node=demo-master-1
INFO[2023-04-14T17:38:52+02:00] 10.62.0.3:22: error: taint "node-role.kubernetes.io/master" not found

innobead commented

Fixed in 0.3.10