d-r-q/qbit

Slow new trx indexing

Opened this issue · 0 comments

d-r-q commented

Currently Index is implemented using HashMap, so when new facts are indexed, the whole map is copied (i.e. on each commit). It leads to slow commit performance. Reimplement Index using some persistent data structure. It may be persistent map from https://github.com/Kotlin/kotlinx.collections.immutable or old qbit's btree: https://github.com/d-r-q/qbit/blob/btree/src/main/kotlin/qbit/collections/btree.kt

For btree, the are some suggestions

  1. remove rebalancing on delete. Since it will be ever growing data structure
  2. probably, add mutable builder. Not sure that it's possible, look at https://github.com/Kotlin/kotlinx.collections.immutable#persistent-collection-builder-interfaces - probably there will be some hints