atlassian/escalator

Implement leader election mechanism

Closed this issue · 1 comments

Implement Kubernetes leader election mechanism to allow for running multiple Escalator replicas, with only one performing the actual work.

At the moment, if multiple Escalator replicas are deployed, all will perform the scaling and will lead to undesired results.

To record an in-person conversation:

  • Leader election will be a command-line option, off by default, for compatibility reasons.
  • Escalator instances that become the leader and are deposed will shut themselves down. This saves handling all the edge cases, and relies on Kubernetes restarting a Pod it's running in to maintain the HA.
  • In the event that you're running leader election outside a cluster using the --kubeconfig option, Escalator will log a warning to remind you that's not a good idea, given the above point.