ethpandaops/ansible-collection-general

k3s ansible role creates unwanted deployments in kube-system

Closed this issue · 1 comments

The k3s ansible role creates local-path-provisioner and metrics-server deployments in kube-system namespace during installation.

These tools are deployed by argocd, so they are sort of redundant.
The metrics-server keeps crashing due to invalid permissions:

W0713 07:57:56.436072       1 reflector.go:324] pkg/mod/k8s.io/client-go@v0.23.2/tools/cache/reflector.go:167: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:serviceaccount:kube-system:metrics-server" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0713 07:57:56.436112       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.23.2/tools/cache/reflector.go:167: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:serviceaccount:kube-system:metrics-server" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

The local-path provisioner in the cloud namespace that gets created by argocd fails, probably due to the other local-path-provisioner created by the k3s role.

Failed sync attempt to a6e54246622863096006e150e06edfa9435db3d9: one or more objects failed to apply, reason: StorageClass.storage.k8s.io "local-path" is invalid: provisioner: Forbidden: updates to provisioner are forbidden. (retried 5 times).

https://argocd.core.ethpandaops.io/applications/k3s-berlin-local-path-provisioner?view=tree&resource=&conditions=false&node=storage.k8s.io%2FStorageClass%2F%2Flocal-path%2F0

Would be good to have a flag of what tools should be deployed into the kube-system during installation.

We disable metrics-server and local-storage on k3s side.

k3s_server_extra_args: >-
...
  --disable=metrics-server
  --disable=local-storage