digitalocean/digitalocean-cloud-controller-manager

Prevent duplicate do-loadbalancer-name annotation from changing LB ownership

jnoordsij opened this issue · 0 comments

jnoordsij commented

Yesterday, I created a second DOKS cluster while the first one had network-traffic issues due to a bug in the latest version after upgrading. When installing my Traefik chart on the second cluster, I forgot to first change the service.beta.kubernetes.io/do-loadbalancer-name annotation on the service.

What happened was that this LB, which was still connected to the (non-functioning) service on the old cluster, was moved to the new cluster service. A little while later, when the issue on the old cluster was resolved, the service on that cluster started running again and "reclaimed" ownership of the LB again.

Altough in this case it was somewhat convenient to have the LB move around so easily, so that traffic was redirected without need to alter DNS records, the sudden changes in traffic (re)direction did cause some confusion on our side.

What I expected to happen was an error preventing me from creating a service with a LB name already in use. According to the docs about changing ownership, this should only work when one first explicitly disowns the LB from the previous service and explicitly provides a kubernetes.digitalocean.com/load-balancer-id when installing/updating the service.

Important note: I do not have a kubernetes.digitalocean.com/load-balancer-id in my chart, so the change of ownership was caused by just the service.beta.kubernetes.io/do-loadbalancer-name tag being identical.

Suggested steps for reproduction:

  1. Create a DOKS Cluster foo
  2. Add a Service/LoadBalancer template to cluster foo with annotation service.beta.kubernetes.io/do-loadbalancer-name: foo
  3. Create a second DOKS Cluster bar
  4. Add a Service/LoadBalancer template to cluster bar with annotation service.beta.kubernetes.io/do-loadbalancer-name: foo (possibly requires a service malfunctioning on cluster foo while setting?)
  5. The LoadBalancer created by the service on cluster foo will move to service on cluster bar.

Expected result (at least for me): step 4. would yield an error: "Cannot provision LoadBalancer: name foo already in use.".