Do not write the entire data set on each request
rightfold opened this issue · 1 comments
rightfold commented
This is obviously inefficient and bad. We should instead write periodically, by having a queue of commands instead of operations, where a command is either an operation or "sync". A timer posts "sync" commands every N seconds.
If we also want to guarantee durability (we currently don't), then we want requests to block until the operation has succeeded, and we should append operations to a journal (we don't want requests to wait for the periodic sync). The sync command would then clear the journal when writing the data to disk. Restarting Icepeak will restore from the journal.