gobanos/cargo-aoc

Allow overriding criterion configs

pedantic79 opened this issue · 0 comments

On Advent of Code 2020, Day 15, part 2. I'm trying to benchmark things in the 1s range in performance

runner: 1.159830989s

With the default criterion config that produces an estimate of:

Benchmarking Day15 - Part2/alt: Collecting 100 samples in estimated 6081.4 s (5050 iterations)

This is quite a long time to wait. Ideally, we would be able to annotate the individual test to override the default criterion config.

I'm able to work around this issue by editing target/aoc/aoc-autobench/benches/aoc_benchmark.rs and changing:

criterion_group!(benches, aoc_benchmark);

into

criterion_group! {
    name = benches;
    config = Criterion::default().significance_level(0.05).sample_size(15);
    targets = aoc_benchmark
}