kubekey安装K8总是遭遇calico controller和coredns起不来

[lqdflying]

What is version of KubeKey has the issue?

v3.0.13

What is your os environment?

debian 10.13 Debian 11.9

KubeKey config file

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: liuqd-k8
spec:
  hosts:
  - {name: k8-master1, address: 192.168.31.91, internalAddress: 192.168.31.91, user: liuqd, privateKeyPath: "~/.ssh/id_rsa"}
  - {name: k8-master2, address: 192.168.31.92, internalAddress: 192.168.31.92, user: liuqd, privateKeyPath: "~/.ssh/id_rsa"}
  - {name: k8-node1, address: 192.168.31.94, internalAddress: 192.168.31.94, user: liuqd, privateKeyPath: "~/.ssh/id_rsa"}
  - {name: k8-node2, address: 192.168.31.95, internalAddress: 192.168.31.95, user: liuqd, privateKeyPath: "~/.ssh/id_rsa"}
  roleGroups:
    etcd:
    - k8-master1
    - k8-master2
    control-plane:
    - k8-master1
    - k8-master2
    worker:
    - k8-node1
    - k8-node2
  controlPlaneEndpoint:
    ## Internal loadbalancer for apiservers 
    # internalLoadbalancer: haproxy

    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.23.17
    clusterName: cluster.local
    masqueradeAll: false
    maxPods: 110
    nodeCidrMaskSize: 24
    proxyMode: ipvs
  network:
    plugin: calico
    calico:
      ipipMode: Always
      vxlanMode: Never
      vethMTU: 1440
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
  registry:
    privateRegistry: ""
    namespaceOverride: ""
    registryMirrors: ["https://harbor.022010.xyz"]
    insecureRegistries: []
  addons: []

A clear and concise description of what happend.

calico controller和coredns起不来,报错

Relevant log output

liuqd@k8-master1:~$ kubectl logs coredns-b7c47bcdc-f2pb7 -n kube-system
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[WARNING] plugin/kubernetes: starting server with unsynced Kubernetes API
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.8.6
linux/amd64, go1.17.1, 13a9191
[INFO] plugin/ready: Still waiting on: "kubernetes"
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:46613->1.1.1.1:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:50988->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:40261->8.8.4.4:53: i/o timeout
[INFO] plugin/ready: Still waiting on: "kubernetes"
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:34060->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:42459->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:44677->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:39316->8.8.4.4:53: i/o timeout
[INFO] plugin/ready: Still waiting on: "kubernetes"
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:59007->1.1.1.1:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:36837->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 6490782168495336201.1480287663792715021. HINFO: read udp 10.233.123.3:60120->8.8.4.4:53: i/o timeout
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
liuqd@k8-master1:~$

Additional information

• 10.0.13的kk配合debian10.13安装v1.23.17的K8,calico controller和coredns根本起不来,
• 10.0.13的kk配合debian11.9安装v1.23.17,最终所有的pod倒是可以起来,但是查看coredns的log依然有很多的HINFO: read udp 10.233.123.3:59007->1.1.1.1:53: i/o timeout,而且创建的pod分配的IP是对的,但是网关和路由不对.
  最后就是日志里经常有

utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.10.0.10]; the provided value is: [169.254.25.10]