Support descending query order
scouten opened this issue · 5 comments
Using tributary with a query such as this …
q = from c in Chunk, order_by: [desc: c.sequence]
… and initial_key set to one greater than the last known sequence number finishes a little more quickly than I'd like. 😉
Tributary is small enough that I've simply inlined the code and hacked on it for my own purposes, so these are not blockers for me in any way, but a version that included this might be helpful for others.
I have to think about descending order for streaming. It's somewhat awkward. Can you give me a use case to work with?
Can't explain deeply without going into proprietary stuff, but the gist of it is I need to process potentially millions of records in reverse order with some stop conditions that are not easily expressed in SQL. Also, I don't need to retain much of the data after processing each individual record (similar to your 1099 case), so streaming seems like the most memory-efficient way to approach this problem.
Hmm, ok. I don't need your specific use case, I just need a use case I can apply to make sure I'm accomplishing the right feature. Or a failing test example
If I could run tests locally 😉 I'd happily construct a failing test. And, as I said before, I'm not at all blocked on anything. So follow up as you can. I'm watching the repo, so I should find out whenever things are updated.