Explore faster database options
awnumar opened this issue · 1 comments
awnumar commented
BoltDB is quite simply brilliant, but its write-speeds are not adequate.
Some alternatives include:
- https://github.com/syndtr/goleveldb/
- https://github.com/siddontang/ledisdb
- https://github.com/HouzuoGuo/tiedot
- https://github.com/peterbourgon/diskv
Speed is the primary concern. Usability is important too, and it should be pure-go. I'm leaning towards LevelDB at the moment.
awnumar commented
It turns out BoltDB has a few other properties that make it unusable for this project. Particularly, very, very, high RAM usage when writing (actually multiple times larger than the size of the data being written) because of its memory-mapping. Also, write speed is inversely proportional to the size of the database.
Therefore we're ditching it in favour of LevelDB, which uses LSM trees instead of b-trees. It doesn't use memory-mapping and is much more constant in terms of speed.