hashicorp/raft

Idea: Atomic swap of node suffrage

Closed this issue · 2 comments

f3nry commented

How difficult would it be to create a new configurationChangeRequest that allowed the suffrage of two nodes to be swapped? It seems like this would be a relatively easy extension to the existing configuration change APIs.

The goal is to create a mechanism by which the consensus levels for a raft cluster can be maintained at exactly N while rolling the cluster gracefully to update the node binaries. The mechanism would utilize the existing AddNonvoter APIs to introduce the new node, but then need a new SwapVoter operation that changes the suffrage of two nodes in a single atomic operation applied to the raft log.

If this sounds reasonable, I may propose the change in a PR.

Thanks!

freeekanayaka commented

It can't be done atomically, afaiu. Only as 2-step operation. That's because if you change 2 nodes at the same time you could have a split brain (it's possible to have 2 separate majorities, one with the old and one with the new configuration, which elect distinct leaders).

  • 🎉1
schristoff commented

Hey @f3nry - thank you for this suggestion. However, we are looking to update our process to reflect more closely what is outlined in the Raft Paper (Section 6 - Cluster Membership Changes) with use of a staging server.
I am going to close this issue for now, but feel free to continue this discussion. :)