sheharyarn/memento

Data replication across nodes with Peerage

sheharyarn opened this issue · 1 comments

sheharyarn commented

Question sent to me via email:

How would I guarantee that Memento/Mnesia is replicating data across nodes, I’m using something called peerage which does node discovery, so at boot the nodes may not all be discovered.

sheharyarn commented

As long as Mnesia knows about the nodes, it will take care of synchronization and data replication automatically. If you want to force data synchronization across all nodes with a transaction, use: Transaction.execute_sync/2.

See these links for more information:

But the core issue of your question is how to add new nodes to Mnesia when you discover them. While Memento does not implement helper functions for that yet (feel free to send in a PR!), you can drop down to :mnesia and accomplish that. You would need to do that in two parts:

1. On the First Node in Cluster

Create Mnesia schema and table if it does not exist. You want to use disc_copies here if that's applicable to you.

2. When a new node joins a Cluster

a. Start Memento/Mnesia on the new node if it hasn't already been started:

Memento.start

b. Tell Mnesia about the new node:

:mnesia.change_config(:extra_db_nodes, [node])

c. Update to use disc copies:

:mnesia.change_table_copy_type(:schema, node, :disc_copies)

d. Copy Table data (Might take a long time depending on the amount of data):

:mnesia.add_table_copy(YourTable, node, :disc_copies)

After this the new node's table would be perfectly synchronized with the rest of the cluster.

Other Resources:

  • 👍5