Support for parallel exports
labianchin opened this issue · 2 comments
labianchin commented
Introduce support for running a single export as multiple SQL queries.
There might be multiple ways to support that:
- User provides a way to split the exports. e.g. split
idcolumn on ranges[0, 10000000],[10000001, 20000000], ... - User specify parallelism, automatically detect splits. e.g.
SELECT min(id) as min_id, max(id) as max_id.
anish749 commented
I’ve started working on this one. Would share a wip PR soon !!