Make record deletion recoverable with BobAliensRecovery
Closed this issue · 0 comments
Currently, if node is not available during deletion, the data on it would not be marked as deleted even after its recovery. In this case, the data in the cluster becomes inconsistent: the record on one replica is deleted, but not on the other. We need to support recovery.
We can do it by making deletion explicitly visible for Bob
and always write a special deletion record to the active blob in aliens. Pearl
should return the deleted
status when trying to read a deleted record, and Bob
should stop scanning older timestamps if it receives that status. So BobAliensRecovery
will recover blobs from aliens that contains that deleted record and Bob
stop scanning if it sees it.
To achieve this on the Pearl
side we should introduce deleted
flag on the index level. This will allow deletion to be reported without reading the data from the blob, thus quickly returning deleted
status. Additionally, deleted
status on index level will simplify blob compaction feature in future