superfly/litefs

Cache checksums for contiguous blocks of pages

Opened this issue · 0 comments

LiteFS maintains an in-memory map of checksums for each page in the database and then aggregates them on commit for each transaction. This is decently fast but can become a bottleneck for larger databases (e.g. 5GB+) with small transactions. To improve this, a higher-level checksum cache can be used to preaggregate blocks of, say, 64 pages at a time. A block in this higher-level cache will be invalidated when a write occurs within its range.