Venemo/node-lmdb

Automatic map resizing

hjerabek opened this issue · 3 comments

hjerabek commented

I came across another LMDB wrapper called lmdb-js. It uses a modified version of mdb.c which allows automatic map resizing. It seems that this feature would only require a few additional lines of code:

https://github.com/kriszyp/lmdb-js/blob/af6e2b74f57bd07aa11846234c563f356aac4755/dependencies/lmdb/libraries/liblmdb/mdb.c#L3461

IMO the same could be implemented here without breaking anyone's code. Was it deliberately not implemented to keep this wrapper as low-level and unbiased as possible?

Venemo commented

I think this feature didn't exist yet (or was considered experimental) when I wrote this module.

kriszyp commented

Was it deliberately not implemented to keep this wrapper as low-level and unbiased as possible?

Yes, and so node-lmdb could use "stock" unmodified LMDB. This change actually involved a fair bit more than just these lines of code. There are also modifications to LMDB to handle MDB_RESIZE events, and tracking previous memory maps and cleaning them up later, so old memory maps from prior snapshots would work during resizes as well.

hjerabek commented

Thanks for the clarifications.