/raft_fleet

A fleet of Raft consensus groups

Primary LanguageElixirMIT LicenseMIT

RaftFleet

Elixir library to run multiple Raft consensus groups in a cluster of Erlang VMs

Hex.pm Build Status Coverage Status

Feature & Design

  • Easy hosting of multiple "cluster-wide state"s
  • Reasonably scalable placement of processes for multiple Raft consensus groups
    • consensus member processes are distributed to ErlangVMs in a data center-aware manner using randezvous hashing
    • automatic rebalancing on adding/removing nodes
  • Each consensus group leader is accessible using the name of the consensus group (which must be an atom)
    • Actual pids of consensus leader processes are cached in a local ETS table for fast access
  • Flexible data model (defined by rafted_value)

Example

Suppose we have a cluster of 4 erlang nodes:

$ iex --sname 1 -S mix
iex(1@skirino-Manjaro)>

$ iex --sname 2 -S mix
iex(2@skirino-Manjaro)> Node.connect :"1@skirino-Manjaro"

$ iex --sname 3 -S mix
iex(3@skirino-Manjaro)> Node.connect :"1@skirino-Manjaro"

$ iex --sname 4 -S mix
iex(4@skirino-Manjaro)> Node.connect :"1@skirino-Manjaro"

Load the following module that implements RaftedValue.Data behaviour on all nodes in the cluster.

defmodule JustAnInt do
  @behaviour RaftedValue.Data
  def new, do: 0
  def command(i, {:set, j}), do: {i, j    }
  def command(i, :inc     ), do: {i, i + 1}
  def query(i, :get), do: i
end

Call RaftFleet.activate/1 on all nodes.

iex(1@skirino-Manjaro)> RaftFleet.activate("zone1")

iex(2@skirino-Manjaro)> RaftFleet.activate("zone2")

iex(3@skirino-Manjaro)> RaftFleet.activate("zone1")

iex(4@skirino-Manjaro)> RaftFleet.activate("zone2")

Create 5 consensus groups each of which replicates an integer and has 3 consensus members.

iex(1@skirino-Manjaro)> rv_config = RaftedValue.make_config(JustAnInt)
iex(1@skirino-Manjaro)> RaftFleet.add_consensus_group(:consensus1, 3, rv_config)
iex(1@skirino-Manjaro)> RaftFleet.add_consensus_group(:consensus2, 3, rv_config)
iex(1@skirino-Manjaro)> RaftFleet.add_consensus_group(:consensus3, 3, rv_config)
iex(1@skirino-Manjaro)> RaftFleet.add_consensus_group(:consensus4, 3, rv_config)
iex(1@skirino-Manjaro)> RaftFleet.add_consensus_group(:consensus5, 3, rv_config)

Now we can run query/command from any node in the cluster:

iex(1@skirino-Manjaro)> RaftFleet.query(:consensus1, :get)
{:ok, 0}

iex(2@skirino-Manjaro)> RaftFleet.command(:consensus1, :inc)
{:ok, 0}

iex(3@skirino-Manjaro)> RaftFleet.query(:consensus1, :get)
{:ok, 1}

Adding/removing nodes triggers rebalancing of consensus member processes.