overridePatches does not work as described
runningman84 opened this issue · 6 comments
my config looks like this:
talosVersion: v1.6.7
endpoint: https://10.0.30.80:6443
additionalApiServerCertSans:
- home-cluster-dev.local
- k8s-dev.home.foo.bar
#- 10.0.30.80
# - 10.0.30.81
# - 10.0.30.82
# - 10.0.30.83
additionalMachineCertSans:
- 10.0.30.81
- 10.0.30.82
- 10.0.30.83
allowSchedulingOnMasters: true
nodes:
- hostname: master1
ipAddress: 10.0.30.81
installDisk: /dev/sda
controlPlane: true
disableSearchDomain: true
nameservers:
- 1.1.1.1
- 8.8.8.8
- 8.8.4.4
networkInterfaces:
- interface: enp0s3
# addresses:
# - 10.0.30.81/24
# routes:
# - network: 0.0.0.0/0
# gateway: 10.0.30.1
# mtu: 1500
dhcp: true
vip:
ip: 10.0.30.80
schematic:
customization:
systemExtensions:
officialExtensions:
- siderolabs/gasket-driver
- siderolabs/intel-ucode
- siderolabs/iscsi-tools
- siderolabs/util-linux-tools
- siderolabs/zfs
- hostname: master2
ipAddress: 10.0.30.82
installDisk: /dev/sda
controlPlane: true
disableSearchDomain: true
nameservers:
- 1.1.1.1
- 8.8.8.8
- 8.8.4.4
networkInterfaces:
- interface: enp0s3
# addresses:
# - 10.0.30.82/24
# routes:
# - network: 0.0.0.0/0
# gateway: 10.0.30.1
# mtu: 1500
dhcp: true
vip:
ip: 10.0.30.80
schematic:
customization:
systemExtensions:
officialExtensions:
- siderolabs/gasket-driver
- siderolabs/intel-ucode
- siderolabs/iscsi-tools
- siderolabs/util-linux-tools
- siderolabs/zfs
- hostname: master3
ipAddress: 10.0.30.83
installDisk: /dev/sda
controlPlane: true
disableSearchDomain: true
nameservers:
- 1.1.1.1
- 8.8.8.8
- 8.8.4.4
networkInterfaces:
- interface: enp0s3
# addresses:
# - 10.0.30.83/24
# routes:
# - network: 0.0.0.0/0
# gateway: 10.0.30.1
# mtu: 1500
dhcp: true
vip:
ip: 10.0.30.80
schematic:
customization:
systemExtensions:
officialExtensions:
- siderolabs/gasket-driver
- siderolabs/intel-ucode
- siderolabs/iscsi-tools
- siderolabs/util-linux-tools
- siderolabs/zfs
overridePatches: false
patches:
- |-
- op: add
path: /machine/kubelet/extraMounts
value:
- destination: /var/mnt/tank
type: bind
source: /var/mnt/tank
options:
- bind
- rshared
- rw
# - hostname: worker1
# ipAddress: 10.0.30.84
# installDisk: /dev/nvme1
# controlPlane: false
controlPlane:
kernelModules:
- name: zfs
patches:
- |-
- op: add
path: /cluster/proxy/extraArgs
value:
metrics-bind-address: "0.0.0.0:10249"
- op: add
path: /machine/kubelet/extraArgs
value:
feature-gates: GracefulNodeShutdown=true
rotate-server-certificates: "true"
- op: add
path: /machine/kubelet/extraMounts
value:
- destination: /var/lib/longhorn
type: bind
source: /var/lib/longhorn
options:
- bind
- rshared
- rw
- destination: /var/lib/local-path-provisioner
type: bind
source: /var/lib/local-path-provisioner
options:
- bind
- rshared
- rw
- op: add
path: /cluster/extraManifests
value:
- https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/main/deploy/standalone-install.yaml
- https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
worker:
kernelModules:
- name: zfs
patches:
- |-
- op: add
path: /machine/kubelet/extraArgs
value:
feature-gates: GracefulNodeShutdown=false
rotate-server-certificates: "true"
- op: add
path: /machine/kubelet/extraMounts
value:
- destination: /var/lib/longhorn
type: bind
source: /var/lib/longhorn
options:
- bind
- rshared
- rw
- destination: /var/lib/local-path-provisioner
type: bind
source: /var/lib/local-path-provisioner
options:
- bind
- rshared
- rw
Based on the docs my idea would be that master 3 would get an extra mount. But master3 looks like all other nodes.
One idea is that /machine/kubelet/extraMounts is wrong, but /machine/kubelet/extraMounts/- gives me that error message:
2024/03/23 14:25:19 failed to generate talos config: failure applying rfc6902 patches to talos machine config: add operation does not apply: doc is missing path: "/machine/kubelet/extraMounts/-": missing value
The problem is because of this line:
talhelper/pkg/config/nodeconfigs.go
Line 15 in 6b2278f
The patches in controlPlane are appended to the patches in nodes[]. So the order is flipped. I'll create a new PR to fix this issue. Thank you!
Try out the latest release @runningman84
i tried your new version 2.3.8 but the problem is still there. Master3 only has the tank mount but not the other mounts defined in the control plane patches.
Weird, I just tried it out and it works fine. Make sure the patches are correct, because the one with - needs object value
can you provide me a sample config which just works?
I have, in the PR I linked above shows the example