Base library for persistent key-value stores, 100% compatible with Python dict.
The idea behind this project is to have a common interface for a multitude of persistent key-value stores, easy to use and extend, with some extra built-in features as bonus. Inspired from Datastore and MemDown.
It plays nice with Object Querying libraries like: JMESPath, or JSONPath RW, or ObjectPath.
- the same API that you already know from Python dict
- thread safe updates
- excellent test coverage (>90%)
- 1st limitation: the keys can only be bytes. They are converted from str to bytes for compatibility.
- 2nd limitation: the values are actually also bytes, but a serializer (default is Pickle) converts the data structure into bytes
- there are several serializers available: Pickle, JSON and optional: cbor2, msgpack. You can easily add your own serializer, please check the documentation below.
- 3rd limitation: after the data was serialized, you have to continue using the same serializer. If for some reason, you want to switch the serializer, you can create a new Store with your new serializer and copy all the data from the old Store.
- the Memory Store and LMDB are available by default. Other key-value Stores used to be implemented (DBM, LevelDB, Redis), but are not included anymore.
This project uses Python 3.10+, or pip.
$ pip install stonesfrom stones import stone, LmdbStore
# Create a new persistent dictionary, backed by LMDB
stor = stone('my-db', LmdbStore, serialize='pickle')
# You can use it just like a normal Python dict,
# with the limitation that keys are bytes,
# and the values must be serializable
stor[b's'] = b'whatever'
stor[b'li'] = [True, False, None, -1]
stor[b'tu'] = ('Yes', 'No', 3)
b's' in stor
# True
len(stor)
# 3
stor.keys()
# [b'li', b's', b'tu']
stor.values()
# [[True, False, None, -1], b'whatever', ('Yes', 'No', 3)]
# The data is decoded in place
-1 in stor[b'li']
# True
3 in stor[b'tu']
# True
del stor[b'li']
stor.get(b'li', False)
# False
# The data is persisted
del stor
# Re-create the store, using the same encoder
stor = stone('my-db', LmdbStore, serialize='pickle')
len(stor)
# 2
stor.keys()
# [b's', b'tu']Python:
- TinyDB - awesome lightweight document oriented database
- RocksDict - Key-value storage supporting any Python object
- uCache - Lightweight caching library for Python
- RaRe-Technologies/sqlitedict - Persistent dict backed by sqlite3 and pickle, multithread-safe
- Lukedeo/Cupboard - store things in a variety of NoSQL KV stores
- ShuhaoWu/Kvkit - unmaintained since 2014
- Datastore - unmaintained since 2014
- Persistent dict ActiveState recipe - from 2009
Node.js:
- lowDB - simple to use local JSON database. Powered by plain JS.
- Keyv - Key-value storage across multiple backends via storage adapters. Supports TTL based expiry, making it suitable as a cache or a persistent key-value store.
- MemDown - abstract LevelDown store for Node.js and browsers
- AppStorage - LocalStorage for Node.js, auto syncing objects with JSON files
- NeDB - embedded database for Node.js, nw.js, Electron and browsers
- FortuneJS - database abstraction layer with a common interface for databases
- PouchDB - pocket-sized database
- ToiletDB - flushes an object to a JSON file
- whatDB - in-memory JavaScript database for Small Web applications that persists to JSON files
Other:
- CubDB - Embedded key-value database written in the Elixir language. It runs locally, it is schema-less, and backed by a single file.
MIT © Cristi Constantin.