Is it a good idea that leader's replicated log from memory not disk?

[vision9527]

// setNewLogs is used to setup the logs which should be appended for a request.
func (r *Raft) setNewLogs(req *AppendEntriesRequest, nextIndex, lastIndex uint64) error {
	// Append up to MaxAppendEntries or up to the lastIndex
	req.Entries = make([]*Log, 0, r.conf.MaxAppendEntries)
	maxIndex := min(nextIndex+uint64(r.conf.MaxAppendEntries)-1, lastIndex)
	for i := nextIndex; i <= maxIndex; i++ {
		oldLog := new(Log)
		if err := r.logs.GetLog(i, oldLog); err != nil {
			r.logger.Error("failed to get log", "index", i, "error", err)
			return err
		}
		req.Entries = append(req.Entries, oldLog)
	}
	return nil
}

[schristoff]

Howdy @vision9527,
Just to clarify, are you asking if we should be replicating while we write to disk? If so, this is referenced as an improvement in Diego's Thesis under section 5.3

The leader appends the command to its log as a new entry, then issues AppendEntries RPCs **in parallel** to each of the other servers to replicate the entry

This is definitely an enhancement I'd be willing to review a PR for - but I'd like to point out that this would be a large change and need a good amount of testing to make sure there are no unintended side effects.

[ncabatoff]

Looks like this got dropped. Sorry about that @vision9527 . If you'd like to take another stab at it we'll try to review more quickly, but do note as @schristoff said that this will probably need a lot of testing.