Add log storage

[mransan]

Logs can be stored permanently by the client application. This is needed in order to be able to handle a very large number of logs.

One possible design which would keep this library pure computation is to modify the LogInterval type used in the global_cache.

Current version

message LogInterval {
  required int32    prev_index      = 1; 
  required int32    prev_term       = 2; 
  repeated LogEntry rev_log_entries = 3;  
  required int32    last_index      = 4;
}

Proposition

message LogInterval {
  required int32    prev_index      = 1; 
  required int32    prev_term       = 2; 
  required int32    last_index      = 4;
  message Compacted { } 
  message Expanded   {
    repeated LogEntry rev_log_entries = 1;  
  }
  oneof data {
    Compacted compacted = 5;
    Expanded   expanded   = 6;
  }
}

The client application would then be able to compact certain log interval. The major difficulty is in the recovery scenario (ie when a follower is lagging behind drastically). When computing the AppendEntries request the leader will find out tha the logs have been compacted. One possible solution is to augment the interface with the application to indicate that loading the data should happen.