Issues with DNS GLB setup
MaxtronGaming opened this issue · 5 comments
Hi,
I followed the instructions on ReadTheDocs on how to setup a DNS only GLB, which resulted in the following config:
[cluster] # All cluster related documentation
name = "MY_GLB_POOL" # Cluster name for all nodes
[cluster.binding]
name = "ns01.example.org" # Our local node name
addr = "127.0.0.1:9000" # Our local binding ip/port
authkey = "test" # key other clusters need to use to talk to us
[[cluster.nodes]]
name = "ns02.example.org" # Secondary cluster nodes name
addr = "6.7.8.9:9000" # Secondary cluster node ip/port binding
authkey = "test" # Key we need to use to talk to the remote node
[dns] # dns configuration
binding = "0.0.0.0" # local binding address for dns service
port = 53 # local binding port for dns service
[dns.domains."glb.example.org"] # config for domain 'domain.nl'
ttl = 11 # set default ttl to 11 seconds
[loadbalancer.settings] # generic load balancer settings
default_balance_method = "topology,firstavailable" # default balance method for all backends
[loadbalancer.pools.INTERNAL_VIP.listener] # define a pool with the name 'INTERNAL_VIP' with an empty listener
[loadbalancer.pools.INTERNAL_VIP.backends.myapp] # in the pool 'INTERNAL_VIP' add a backend with the name 'myapp'
hostnames = ["update.glb.example.org"] # the backend called 'myapp' listens to the hostname 'www.glb.example.org'
connectmode="http" # we connect to the backends using http protocol
[loadbalancer.pools.INTERNAL_VIP.backends.myapp.balance]
method = "topology,firstavailable"
[loadbalancer.pools.INTERNAL_VIP.backends.myapp.dnsentry] # add a GLB dns entry that will register the domain if this backend is online
domain = "glb.example.org" # domain name this GLB entry belongs to
hostnames = "update" # hostname to add when this backend is online
ip = "3.4.5.6" # ip to point this record to if it is online
[loadbalancer.pools.INTERNAL_VIP.backends.myapp.healthcheck] # define a healthcheck for the backend called 'myapp'
type = "tcpconnect" # use a tcpconnect to test if the backend is up
[[loadbalancer.pools.INTERNAL_VIP.backends.myapp.nodes]] # add a backend node to backend of 'myapp'
hostname = "webserver1" # friendly name of the host
ip = "1.2.3.4" # ip address of the backend host to serve requests to
port = 80 # port of the backend host to serve requests to
[[loadbalancer.pools.INTERNAL_VIP.backends.myapp.nodes]] # add a second backend node to backend of 'myapp'
hostname = "webserver2" # friendly name of the host
ip = "2.3.4.5" # ip address of the backend host to serve requests to
port = 80
But when I run a DNS query against the cluster, I don't get a response and this log output:
time="2022-01-18T13:43:32Z" level=debug msg=Client client=5.7.3.1 clientdata=5.7.3.1 func=dns orgclient="5.7.3.1:58714" tag=dns/server/parse
time="2022-01-18T13:43:32Z" level=debug msg="Non local zone request" domain=update.glb.example.org fqdn=update.glb.example.org. func=dns querytype=A tag=dns/server/parse
time="2022-01-18T13:43:32Z" level=info msg="DNS request from client" 0x20=false client=5.7.3.1 domain=glb.example.org func=dns hostname=update querytype=A tag=dns/server/parse
time="2022-01-18T13:43:32Z" level=debug msg="Matching dns records" domain=glb.example.org func=dns name=update online=0 records=0 tag=dns/server/getallrecords type=A
time="2022-01-18T13:43:32Z" level=debug msg="Matching dns records" domain=glb.example.org func=dns name=update online=0 records=0 tag=dns/server/getallrecords type=CNAME
time="2022-01-18T13:43:32Z" level=debug msg="Request Finished" exitcode=2 func=dns tag=dns/server/parse
Hello,
the dnsentry block should have its hostname defined as hostname not hostnames
this would work better:
[loadbalancer.pools.INTERNAL_VIP.backends.myapp.dnsentry] # add a GLB dns entry that will register the domain if this backend is online
domain = "glb.example.org" # domain name this GLB entry belongs to
hostname = "update" # hostname to add when this backend is online
ip = "3.4.5.6" # ip to point this record to if it is online
you can see in the UI (127.0.0.1:9001) when you click on the GLB tab, that in the old config its missing the hostname, and only has the domain name as dns entry, and in the above example, it stats update.glb.example.org, which then is resolvable
and yes on a side note, it should properly alert when you specify no 'hostname', that would have made your troubleshooting easier. I'll have a look at that. but for now this should solve your issue.
Hey,
thank you for your swift reply. That fixed it for me. I want to mention here that this a error in the documentation then?
Also what IP do i have to put here? Because I have 3 globaly distributed nodes, that I want to globaly loadbalance, as in Mercury responds with the node closest to the client.
[loadbalancer.pools.INTERNAL_VIP.backends.myapp.dnsentry] # add a GLB dns entry that will register the domain if this backend is online
domain = "glb.example.org" # domain name this GLB entry belongs to
hostname = "update" # hostname to add when this backend is online
ip = "3.4.5.6" # ip to point this record to if it is online
Hello,
that dnsentry ip will be the public ip where your client can reach Mercury.
so if you have 3 nodes, and the public ip (or that of a firewall in front) is 1.2.10.1, 1.2.20.1 and 1.2.30.1
then your 3 mercury nodes will each have the correct public ip configured as dns entry
when a client then resolves the dns entry (update.glb.example.com) they will connect to this ip + port of the configured listener
one note though, you mention you want clients to connect to the closest Mercury node. this is not something Mercury can do automaticly.
you can do this using topology based loadbalancing (which is based on ip ranges) with this you can let mercury give a higher priority to the node which matches the configured ranges, but this is a manual configuration. (for example if you state that node 1 has a 'local_topology' configured with '1.2.10.0/24' then if you resolve the dns entry on another node, it wil show node 1's ip as first entry)
the result however depends on your dns servers:
lets say you have external dns servers, and they resolve the mercury in order of node1, node2, node3, then the round robin methods of that dns server might shuffle these ip's.
a 'trick' to bypass that if needed is to also loadbalance using 'firstavailable' so: (balance.method: 'topology,firstavailable')
the result is that only the preferred ip would appear (its on the top of the list, and the only one returned to the dns server)
keep this in mind when trying to attempt the above, knowing that its implementation is not so traight forward, especially if you want to go down the path of adding public ip ranges based on geographical locations, you probably need something more advanced :)
Hey,
thanks again for your help and time.
I guess i messed up the description of my situation slighty. I currently have two nodes that run Mercury and three nodes that server HTTP requests. I want to use the two Mercury nodes as DNS servers, which do a global loadbalancing between the three webnodes, on which i don't intend to run Mercury as a proxy.
I also wonder how I have to manage the config files if I have multiple nodes? If I run my current configs and try to resolve the hostname, I get both IPs despite having set the firstavailabe balance mode.
Here are my two config files:
Node 1:
[cluster] # All cluster related documentation
name = "UPDATE_SERVICES" # Cluster name for all nodes
[cluster.binding]
name = "ns01.example.org" # Our local node name
addr = "0.0.0.0:9000" # Our local binding ip/port
authkey = "example" # key other clusters need to use to talk to us
[[cluster.nodes]]
name = "ns02.example.org" # Secondary cluster nodes name
addr = "1.2.3.4:9000" # Secondary cluster node ip/port binding
authkey = "example" # Key we need to use to talk to the remote node
[dns] # dns configuration
binding = "0.0.0.0" # local binding address for dns service
port = 53 # local binding port for dns service
[dns.domains."glb.example.org"] # config for domain 'example.org'
ttl = 1 # set default ttl to 11 seconds
[loadbalancer.settings] # generic load balancer settings
default_balance_method = "topology,firstavailable" # default balance method for all backends
[loadbalancer.pools.CLUSTER01.listener] # define a pool with the name 'CLUSTER01' with an empty listener
[loadbalancer.pools.CLUSTER01.backends.updateservice] # in the pool 'CLUSTER01' add a backend with the name 'updateservice'
hostnames = ["update.glb.example.org"] # the backend called 'updateservice' listens to the hostname 'www.example.com'
connectmode="https" # we connect to the backends using https protocol
[loadbalancer.pools.CLUSTER01.backends.updateservice.balance]
method = "topology,firstavailable"
[loadbalancer.pools.CLUSTER01.backends.updateservice.dnsentry] # add a GLB dns entry that will register the domain if this backend is online
domain = "glb.example.org" # domain name this GLB entry belongs to
hostname = "update" # hostname to add when this backend is online
ip = "5.6.7.8" # ip to point this record to if it is online
[loadbalancer.pools.CLUSTER01.backends.updateservice.healthcheck] # define a healthcheck for the backend called 'updateservice'
type = "tcpconnect" # use a tcpconnect to test if the backend is up
[[loadbalancer.pools.CLUSTER01.backends.updateservice.nodes]] # add a backend node to backend of 'updateservice'
hostname = "eu01" # friendly name of the host
ip = "5.6.7.8" # ip address of the backend host to serve requests to
port = 443 # port of the backend host to serve requests to
[[loadbalancer.pools.CLUSTER01.backends.updateservice.nodes]] # add a second backend node to backend of 'updateservice'
hostname = "us01" # friendly name of the host
ip = "6.7.8.9" # ip address of the backend host to serve requests to
port = 443
Node 2:
[cluster] # All cluster related documentation
name = "UPDATE_SERVICES" # Cluster name for all nodes
[cluster.binding]
name = "ns02.example.org" # Secondary cluster nodes name
addr = "0.0.0.0:9000" # Secondary cluster node ip/port binding
authkey = "example" # Key we need to use to talk to the remote node
[[cluster.nodes]]
name = "ns01.example.org" # Our local node name
addr = "2.3.4.5:9000" # Our local binding ip/port
authkey = "example" # key other clusters need to use to talk to us
[dns] # dns configuration
binding = "0.0.0.0" # local binding address for dns service
port = 53 # local binding port for dns service
[dns.domains."glb.example.org"] # config for domain 'example.org'
ttl = 1 # set default ttl to 11 seconds
[loadbalancer.settings] # generic load balancer settings
default_balance_method = "topology,firstavailable" # default balance method for all backends
[loadbalancer.pools.CLUSTER01.listener] # define a pool with the name 'CLUSTER01' with an empty listener
[loadbalancer.pools.CLUSTER01.backends.updateservice] # in the pool 'CLUSTER01' add a backend with the name 'updateservice'
hostnames = ["update.glb.example.org"] # the backend called 'updateservice' listens to the hostname 'www.example.com'
connectmode="https" # we connect to the backends using https protocol
[loadbalancer.pools.CLUSTER01.backends.updateservice.balance]
method = "topology,firstavailable"
[loadbalancer.pools.CLUSTER01.backends.updateservice.dnsentry] # add a GLB dns entry that will register the domain if this backend is online
domain = "glb.example.org" # domain name this GLB entry belongs to
hostname = "update" # hostname to add when this backend is online
ip = "6.7.8.9" # ip to point this record to if it is online
[loadbalancer.pools.CLUSTER01.backends.updateservice.healthcheck] # define a healthcheck for the backend called 'updateservice'
type = "tcpconnect" # use a tcpconnect to test if the backend is up
[[loadbalancer.pools.CLUSTER01.backends.updateservice.nodes]] # add a backend node to backend of 'updateservice'
hostname = "eu01" # friendly name of the host
ip = "5.6.7.8" # ip address of the backend host to serve requests to
port = 443 # port of the backend host to serve requests to
[[loadbalancer.pools.CLUSTER01.backends.updateservice.nodes]] # add a second backend node to backend of 'updateservice'
hostname = "us01" # friendly name of the host
ip = "6.7.8.9" # ip address of the backend host to serve requests to
port = 443
Well I have opted to use Cloudflare so I'm going to close this.