Is there a fast way to get all keys available on the DB
orena1 opened this issue · 2 comments
orena1 commented
The only method that I say is:
list(Rdict(f'...DBS/rocksdict/{date}',options=opt).keys())
but it takes a lot of time, from what I understand it iterate over all the data.
Thanks,
Congyuwang commented
I think this already is the fastest way. Use raw mode too.
Congyuwang commented
All data have to be retrieved from the disk, since this is by design what rocksdb does. It assumes that there is difficulty storing everything inside main memory.
However, from what I can see, in your use case, there is no difficulty keeping all keys within main memory. So, what you can do is perhaps just keep all your keys in memory, and also perhaps separately deserialize all your keys in another format for fast batch retrieval.