edoliberty/streaming-quantiles

How to set k?

Titanssword opened this issue · 1 comments

Hi,
I am a little bit confused about this.
If I want to set the ε to 0.01, which means that | R̃(x) − R(x)| ≤ 0.01 * n. What value should k be set?
Is there some certain connection between k and ε? I am looking forward to your reply.

k should behave like constant/eps.
It's easy to run a few tests to get a handle on the exact constant.
Please feel free to contribute a notebook or python script that does that.

As a whole (like with all streaming algorithms) the size parameter (k) and the accuracy parameter (eps) are "opposing each other". The algorithm can let you specify eps and strive to achieve at most eps error with the smallest size sketch. Or, you can specify the size k and strive to give the most accurate results. Here we chose the latter. With the information above, we can enable both modes.