eBay/HomeStore

[Resync] Baseline resync: old member rejoin

Opened this issue · 0 comments

yamingk commented

old member re-joining after a crash or shutdown for sometime which makes its last committed lsn not in scope of existing quorum, in other words, can't do incremental resync.

Phase-1:
We can still use (#261 ) to resync to this member. Old member needs to format itself as a fresh booted member if it detects (needs some API between leader and follower to get this knowledge) it can't be served as an incremental resync.

Phase-2 (Nice to have?):
Get the last snapshot and get its log index saved in its snapshot and try to get a diff entries from the index btree on leader side:

  • Query the btree to return all the entries whose LSN is larger than the LSN mapped to log index saved in this new member’s last snapshot. Need a full scan of the btree entries. Currently there is no api to do range query based on LSN.

  • Pack scanned entries (only keys are needed) with remaining log entries to the remote side and have the remote side to fetch data based on key.