This crate provides an API to work with immutable (string -> string) maps stored on disk. The main access method are iterators, but there's a simpler API, too.
The general process is
- Writing a table, using
TableBuilder. The entries have to be added in sorted order. The data doesn't have to be written to disk; any type implementingWriteworks. - Reading a table, using
Table. Again, the source is generic; any type implementingRead + Seekcan be used.
Note that the tables and some other structures are generic over the ordering of
keys; usually you can just use StandardComparator, though.
With Options, you can influence some details of how tables are laid out on
disk. Usually, you don't need to; just use the Options::default() value.
If there's data corruption in the files on disk, defective blocks will be
skipped. How many entries a single block contains depends on the block size,
which can be set in the Options struct.
This crate reuses code originally written for the persistence part of rusty-leveldb, a reimplementation of Google's LevelDB in Rust. That's the reason for the code being a bit more complicated than needed at some points.
With no compression on a tmpfs volume running on an idle Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz processor, the benchmark shows that in tables of 10'000
entries of each 16 key bytes and 16 value bytes, this crate will
- read 5.3 million entries per second
- write 1.2 million entries per second
The performance for tables of different sizes may differ.
Checksum verification failures often stem from either corruption (obviously) or incompletely written or half-overwritten SSTable files.
Contributions are very welcome! Feel free to send pull requests.