Migrate from python-rocksdb to rocksdict
wbarnha opened this issue · 0 comments
wbarnha commented
We should give users the option to install rockdict
as an alternative to faust-streaming-rocksdb
for a few reasons:
- I have profiled the performance of
rocksdict
personally and have noticed a 20x boost in performance. There's also an abundance of information on https://github.com/Congyuwang/RocksDict demonstrating its performant abilities rocksdict
is actively maintained, I have not been able to actively supportfaust-streaming-rocksdb
as muchrocksdict
wheels are only 4 MB, whereasfaust-streaming-rocksdb
wheels are 66 MB due to its intensive CI/CD build. This is undesirable because it can increase dependency installation time and Docker image size.
I think adding faust-streaming[rocksdict]
may be the right way to approach this for now until we know beyond a reasonable doubt that we can phase out faust-streaming-rocksdb
.