Refactor the `LogEntryVecDeque` in the log.rs
Closed this issue · 1 comments
After the pr #704 being merged, we got a LogEntryVecDeque
implementation like this:
/// For the leader, there should never be a gap between snapshot and entries
pub(super) struct Log<C: Command> {
/// Log entries, should be persisted
/// Note that the logical index in `LogEntry` is different from physical index
entries: LogEntryVecDeque<C>,
/// The last log index that has been compacted
pub(super) base_index: LogIndex,
/// The last log term that has been compacted
pub(super) base_term: u64,
/// Index of highest log entry known to be committed
pub(super) commit_index: LogIndex,
/// Index of highest log entry sent to after sync. `last_as` should always be less than or equal to `last_exe`.
pub(super) last_as: LogIndex,
/// Index of highest log entry sent to speculatively exe. `last_exe` should always be greater than or equal to `last_as`.
pub(super) last_exe: LogIndex,
/// Contexts of fallback log entries
pub(super) fallback_contexts: HashMap<LogIndex, FallbackContext<C>>,
/// Tx to send log entries to persist task
log_tx: mpsc::UnboundedSender<Arc<LogEntry<C>>>,
/// Entries to keep in memory
entries_cap: usize,
}
struct LogEntryVecDeque<C: Command> {
/// A VecDeque to store log entries, it will be serialized and persisted
entries: VecDeque<Arc<LogEntry<C>>>,
/// entry size of each item in entries
entry_size: VecDeque<u64>,
/// the right index of the batch (offset)
/// batch_range: [i, i + batch_index[i]]
batch_index: VecDeque<usize>,
/// the first entry idx of the current batch window
first_entry_at_last_batch: usize,
/// the current batch window size
last_batch_size: u64,
/// Batch size limit
batch_limit: u64,
}
As you can see, batch_index
stores an offset. With this offset, we can calculate a log batch of size less than or equal to batch_size
for any given index i
(FYI: issue #368 ). We record the offset instead of the right bound in that the log may be compressed. However, there is a method named li_to_pi
in Log
, which can transform the logical index to the physical index (The compression only changes the physical index ):
/// Transform logical index to physical index of `self.entries`
fn li_to_pi(&self, i: LogIndex) -> usize {
assert!(
i > self.base_index,
"can't access the log entry whose index is not bigger than {}, might have been compacted",
self.base_index
);
(i - self.base_index - 1).numeric_cast()
}
Therefore, we think we can do some refactoring stuff: move li_to_pi
and relevant fields, like base_index
and base_term
into LogEntryVecDeque
.
👋 Thanks for opening this issue!
Reply with the following command on its own line to get help or engage:
/contributing-agreement
: to print Contributing Agreements./assignme
: to assign this issue to you.