google/temporian

Feature Request: Moving Quantile Window Operator

Opened this issue · 1 comments

As you might expect, moving quantiles are particularly slow by their nature. Standard python (e.g. Pandas, numpy) libraries offer moving/rolling/expanding quantiles but these are often slow and scale very poorly to 100,000 of data points.

Temporian seems like the perfect place to introduce both language specific speed-ups in C++ but also algorithmically superior operations. Here are some examples:
Heap based moving median: https://aakinshin.net/posts/partitioning-heaps-quantile-estimator/

Approximate moving median:
https://aakinshin.net/posts/mp2-quantile-estimator/
https://aakinshin.net/posts/partitioning-heaps-quantile-estimator2/

Thanks for the proposal @joshdunnlime! Agree that Temporian is a fitting home for this kind of implementation :)

Added it to our backlog and will work on it when we have the bandwidth! If you'd like to contribute to make it happen asap do let us know and we'll arrange it.

Cheers!