Azure KOPS - Node in VMSS having capital letter(Appended by Azure/vmss as postfix) in computerName not joining the cluster

[ajgupta42]

/kind bug

1. What kops version are you running? The command kops version, will display
this information.
Client version: 1.28.4 (git-v1.28.4)

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.7

3. What cloud provider are you using?
Azure

4. What commands did you run? What is the simplest way to reproduce this issue?
Add 11 nodes in an Instance Group/VMSS

5. What happened after the commands executed?
Nodes without capital letter in name join the cluster, last node with name test00000A doesn't join the cluster, cause it do have capital letter in name
Error in Api server logs:
I0417 11:21:21.770390 11 node_authorizer.go:290] NODE DENY: 'test00000A' &authorizer.AttributesRecord{User:(*user.DefaultInfo)(0xc00ffb8580), Verb:"get", Namespace:"", APIGroup:"storage.k8s.io", APIVersion:"v1", Resource:"csinodes", Subresource:"", Name:"test00000a", ResourceRequest:true, Path:"/apis/storage.k8s.io/v1/csinodes/test00000a"}

6. What did you expect to happen?

I want the last node to be authorized & join the cluster.
But as the node name in vmss is in capital letters but nodename in kubernetes is in lowercase, it is having issues

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

apiVersion: kops.k8s.io/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: "2024-04-10T10:52:32Z"
  name: xyz.eastus.azure.reai.io
spec:
  api:
    loadBalancer:
      type: Public
  authorization:
    rbac: {}
  channel: stable
  cloudConfig:
    azure:
      adminUser: xxxxx
      resourceGroupName: xxxxx
      routeTableName: xxxxx
      subscriptionId: xxxxxxx
      tenantId: xxxxxx
  cloudLabels:
    cluster-name: xyz
    k8s.io_cluster-autoscaler_xyz.eastus.azure.reai.io: owned
    k8s.io_cluster-autoscaler_enabled: "1"
    k8s.io_cluster-autoscaler_node-template_label: "1"
  cloudProvider: azure
  configBase: azureblob://cluster-configs/xyz.eastus.azure.reai.io
  etcdClusters:
  - cpuRequest: 200m
    etcdMembers:
    - instanceGroup: control-plane-eastus-3
      name: etcd-3
    manager:
      backupRetentionDays: 7
    memoryRequest: 100Mi
    name: main
  - cpuRequest: 100m
    etcdMembers:
    - instanceGroup: control-plane-eastus-3
      name: etcd-3
    manager:
      backupRetentionDays: 7
    memoryRequest: 100Mi
    name: events
  iam:
    allowContainerRegistry: true
    legacy: false
  kubeControllerManager:
    terminatedPodGCThreshold: 1024
  kubeDNS:
    nodeLocalDNS:
      enabled: true
      provider: CoreDNS
  kubeProxy:
    enabled: true
  kubelet:
    anonymousAuth: false
    authenticationTokenWebhook: true
    authorizationMode: Webhook
  kubernetesVersion: 1.28.7
  masterPublicName: api.xyz.eastus.azure.reai.io
  networkCIDR: x.y.z.a/20
  networking:
    cilium: {}
  nonMasqueradeCIDR: a.b.c.d/10
  subnets:
  - cidr: a.b.c.d/22
    name: utility-eastus
    region: eastus
    type: Public
  - cidr: x.y.z.g/21
    name: eastus
    region: eastus
    type: Private
  topology:
    dns:
      type: None
  updatePolicy: external

8. Please run the commands with most verbose logging by adding the -v 10 flag.
Paste the logs into this report, or in a gist and provide the gist link here.
I0417 11:21:21.770390 11 node_authorizer.go:290] NODE DENY: 'test00000A' &authorizer.AttributesRecord{User:(*user.DefaultInfo)(0xc00ffb8580), Verb:"get", Namespace:"", APIGroup:"storage.k8s.io", APIVersion:"v1", Resource:"csinodes", Subresource:"", Name:"test00000a", ResourceRequest:true, Path:"/apis/storage.k8s.io/v1/csinodes/test00000a"}

9. Anything else do we need to know?

Only the node with capital letters are having problem, it should be changed to lowercase & join the cluster.
I have tried adding 'hostnamectl set-hostname $(hostname | tr [:upper:] [:lower:])' to userdata of vmss in the beginning.
But still facing same error

[hakman]

Thanks you for reporting this @ajgupta42!

[ajgupta42]

@hakman Thanks for fixing this so quickly.
Are we planning for new release sometime soon ?
v1.28.4 was released approx 2.5 months back

[hakman]

There is a new release planned this week or next week.