Not possible to compute the (Ratio, RatioSD)
AndreyAkinshin opened this issue · 1 comments
AndreyAkinshin commented
Discussed in #2584
Originally posted by sastru July 2, 2024
I get the warning
// * Warnings *
BaselineCustomAnalyzer
Summary -> A question mark '?' symbol indicates that it was not possible to compute the (Ratio, RatioSD) column(s) because the baseline value is too close to zero.
I suspect it is because somehow all my benchmarks are separated by a line, why? how can I fix that?
Program.cs
BenchmarkRunner.Run<Benchmarks>(DefaultConfig.Instance
.AddJob(Job.InProcess
.WithStrategy(RunStrategy.Monitoring)
.DontEnforcePowerPlan()
.WithToolchain(new InProcessEmitToolchain(true))
), args);
Benchmarks.cs
...
[Benchmark(Baseline = true)]
[Arguments(1028, 1028, false, OutsideQueryVariation.Default, false, null, 30)]
public async Task Baseline(int limit, int totalCount, bool includeOutsidePoints, OutsideQueryVariation useOutsideQueryVariation, bool useMaxOutsideInterval, int? status, int timeSpanInMin) { .. }
[Benchmark]
[Arguments(1028, 1028, false, OutsideQueryVariation.Default, false, 192, 30)]
[Arguments(1028, 1030, true, OutsideQueryVariation.Default, false, null, 30)]
public async Task Benchmark(int limit, int totalCount, bool includeOutsidePoints, OutsideQueryVariation useOutsideQueryVariation, bool useMaxOutsideInterval, int? status, int timeSpanInMin) { .. }
```</div>
timcassell commented
Let's use this issue to track fixing the warning message. Use #2596 to track improving the usability of baselines with arguments.