mongodb-partners/mongo-rocks

use compaction for oplog capping and truncateAfter

wolfkdy opened this issue · 2 comments

Currently, oplog capping and truncateAfter uses a non-timestamped Delete to delete timestamp history, although wt supports this, rocksdb 6.x series don't support this, I think this is not elegant.
To avoid this, we may use compaction to do oplog capping. For other capped-collections, capping delete is timestamped.
Secondly, this should gain 10%-20% write performance because disk-io is reduced.

  1. remove oplogKeyTracker
  2. disable oplogCF compaction to
    2.1) get numRecords, storageSize from rocksdb's metadata instead of CounterManager
  3. use DeleteFilesInRange + CompactRange + CompactionFilter to purge old oplogs
  4. rewrite reclaimOplog + cappedTrancateAfter
  5. jstests access serverStatus().rocksdb.compactionScheduler to access debugInfo to do sanity check
  6. remove redundent code from RocksRecordStore.cappedDeleteAsNeeded_inlock and compaction scheduler
  7. the compaction apis list in 5) should be in CompactionScheduler, use Notify to inform job completion, jstests should cover the case of shutting down when sending notification

#172 close this.