storage: support range-filter scan by sort key
Closed this issue · 0 comments
Our storage engine already supports sorting by sort key. To leverage the property in storage, we can provide a range_scan
interface on storage, so as to support range filter scan for executors.
To support range_scan, we should be able to start scanning from a sort key in storage engine. This requires two steps:
- record sort key in block index
- support create iterator by sort key
- add interface for executors' use
Currently, the first_key
field is empty in block builder. We should record first_key
in order to support seek. Note that #588 will add composite sort key support. For this issue, we may only support range filter scan by the first element in composite sort key.
risinglight/src/storage/secondary/block/block_index_builder.rs
Lines 39 to 54 in 7c5e0d5
After that, we should be able to support create RowSetIterator
by sort key.
risinglight/src/storage/secondary/rowset/rowset_iterator.rs
Lines 27 to 33 in 7c5e0d5
The interface should have two new fields start_key: Option<Vec<u8>>
and end_key: Option<Vec<u8>>
. RowSetIterator should return rows only in this range. At the same time, the RowSetIterator should directly seek to the block of start_key
instead of scanning from the beginning.
When everything's ready, we can finally expose range-filter scan interface to the compute layer. This requires us to modify:
risinglight/src/storage/secondary/transaction.rs
Lines 356 to 364 in 7c5e0d5