:mag: add benchmarks
Opened this issue · 5 comments
Add benchmarks - that are reproducible - to this repo.
This would also allow contributors to check whether their changes impact (& possibly improve) performance :)
TODOs
- improve Rust benchmarks
- add Python benchmarks
=> try to make the Rust & Python benchmarks comparable - using same arraylength, n_out, dtype configurations
would be super cool to see some comparisons drawn from your paper -- unsure how the following graphs relate but below is a heatmap showing various differences in distance metrics for 250,000 data points and how they stack up against 10% retention (at most 25k points used in the n_out param)
Could also probably do like a png difference? Save the plot file and actually compare the resulting images somehow?
Hi @jayceslesar are you referring to the comparisons from this paper?
If so, I guess we can cross-reference our ts-datapoint-selection-vis repo!
As MinMaxLTTB is in its essence a novel algorithm (2-step algo using first MinMax & then LTTB), we ended up writing a desperate (short) paper about it (and did not include it in the ts-datapoint-selection-vis repo). The preprint of this paper will appear online tomorrow :) The benchmarks of MinMaxLTTB its visual representativeness are in our MinMaxLTTB repo
Could also probably do like a png difference? Save the plot file and actually compare the resulting images somehow?
That is exactly what we do for a matrix difference! We use DSSIM, PEM_20, and MSE to capture this! :)
@jayceslesar our paper on MinMaxLTTB https://arxiv.org/abs/2305.00332