Also compress the active db file
Closed this issue · 0 comments
neonrust commented
Currently, the active db file (series
) is not compressed; only the backups are.
Also compressing the active one has several benefits.
- Reading the compressed file is very fast. With
zstd
, it's negligible. (it it's on a "slow" medium, it's probably faster than uncompressed, but I doubt that's the case nowadays) - When writing the series file we already need to compress 1 file (the .1 backup).
- With "sparse" backups (for #42), we don't need to compress anything.
- Rolling back a backup is essentially a no-op.
Ideally, we should figure out how to reading and decompressing at the same time, i.e. the equivalent of zstd -d -c file.zst | jless
. Or, at the very least, test if that is faster. That being said, orjson
doesn't support reading from a file descriptor, so maybe this would be pointless.
Preferrably, this should be done before #42.