use compaction for oplog capping and truncateAfter
wolfkdy opened this issue · 2 comments
wolfkdy commented
Currently, oplog capping and truncateAfter uses a non-timestamped Delete to delete timestamp history, although wt supports this, rocksdb 6.x series don't support this, I think this is not elegant.
To avoid this, we may use compaction to do oplog capping. For other capped-collections, capping delete is timestamped.
Secondly, this should gain 10%-20% write performance because disk-io is reduced.
wolfkdy commented
- remove oplogKeyTracker
- disable oplogCF compaction to
2.1) get numRecords, storageSize from rocksdb's metadata instead of CounterManager - use DeleteFilesInRange + CompactRange + CompactionFilter to purge old oplogs
- rewrite reclaimOplog + cappedTrancateAfter
- jstests access serverStatus().rocksdb.compactionScheduler to access debugInfo to do sanity check
- remove redundent code from RocksRecordStore.cappedDeleteAsNeeded_inlock and compaction scheduler
- the compaction apis list in 5) should be in CompactionScheduler, use Notify to inform job completion, jstests should cover the case of shutting down when sending notification