host.k3d.internal -> k3d-edgedns-server-0

[jkremser]

Some background:

chart/k8gb/values.yaml:114:  - host: host.k3d.internal
                             - port: 1053

(link)

host.k3d.internal is there for the clusters to be able to talk with the host (overloaded term, in this case i mean our mac)
Currently, it kinda work, because the sevice on localhost:1053 (again mac) listens, because of the nodeport + k3d cfg.

However, wouldn’t it be better if the workload clusters talk directly with the edge cluster? And not via the “back to host route and then go to the edge cluster”.. The Configmap for cluster coredns is actually populated by k3d also with other hostnames for each cluster:

 k get cm coredns -n kube-system -o yaml | grep -A7 NodeHosts:
  NodeHosts: |
    172.19.0.1 host.k3d.internal
    172.19.0.4 k3d-test-gslb1-agent-0
    172.19.0.2 k3d-edgedns-server-0
    172.19.0.3 k3d-test-gslb1-server-0
    172.19.0.6 k3d-test-gslb2-agent-0
    172.19.0.5 k3d-test-gslb2-server-0

I've tried to "talk" to k3d-edgedns-server-0 from both "workload" clusters (test-gslb{1,2}) and it worked well:

k run -it --image=infoblox/dnstools --rm --restart=Never --command debug -- dig +short @k3d-edgedns-server-0 -p 30053 roundrobin.cloud.example.com
172.19.0.3
172.19.0.6
172.19.0.5
172.19.0.4
pod "debug" deleted

It's a minor issue, however with this approach we don't have to expose any ports for edge dns server on host (mac) at all -> if also other nodePorts (for metrics) are gone we can theoretically run multiple sets of clusters on one mac and compare, no port collisions.