topling/mytopling

Add multiple indexes in one pass table scan

Closed this issue · 2 comments

MyRocks inplace_populate_sk() scan table one pass for each index, this waste CPUs, but this waste is not a hot spot because MyRocks MergeTree is very slow -- thus MergeTree is the hot spot.

MyTopling has auto_sort_sst, which is very fast, especially for building indexes, thus the wasted table scans are hot spot.

So creating multiple indexes shoule be realized by just one pass table scan, at least for non-unique index.

Commits:

  • 91362ad 2023-10-21 13:37:11 +0800 inplace_populate_sk: build multi non-unique indexes in one table scan
  • ab22ce2 2023-10-21 10:22:40 +0800 rdb_sst_info: Add m_avg_max_size
  • ca3e8ec 2023-10-20 20:09:52 +0800 scan table just once when building multiple indexes by alter table

This feature had been submited to MyRocks earlier: facebook/mysql-5.6#1376