dotnet/BenchmarkDotNet

Not possible to compute the (Ratio, RatioSD)

AndreyAkinshin opened this issue · 1 comments

Discussed in #2584

Originally posted by sastru July 2, 2024
image

I get the warning

// * Warnings *
BaselineCustomAnalyzer
Summary -> A question mark '?' symbol indicates that it was not possible to compute the (Ratio, RatioSD) column(s) because the baseline value is too close to zero.

I suspect it is because somehow all my benchmarks are separated by a line, why? how can I fix that?

Program.cs

BenchmarkRunner.Run<Benchmarks>(DefaultConfig.Instance
    .AddJob(Job.InProcess
        .WithStrategy(RunStrategy.Monitoring)
        .DontEnforcePowerPlan()
        .WithToolchain(new InProcessEmitToolchain(true))
    ), args);

Benchmarks.cs

...
[Benchmark(Baseline = true)]
[Arguments(1028, 1028, false, OutsideQueryVariation.Default, false, null, 30)]
public async Task Baseline(int limit, int totalCount, bool includeOutsidePoints, OutsideQueryVariation useOutsideQueryVariation, bool useMaxOutsideInterval, int? status, int timeSpanInMin) { .. }

[Benchmark]
[Arguments(1028,  1028,  false, OutsideQueryVariation.Default,  false, 192,  30)]
[Arguments(1028,  1030,  true,  OutsideQueryVariation.Default,  false, null, 30)]
public async Task Benchmark(int limit, int totalCount, bool includeOutsidePoints, OutsideQueryVariation useOutsideQueryVariation, bool useMaxOutsideInterval, int? status, int timeSpanInMin) { .. }
```</div>

Let's use this issue to track fixing the warning message. Use #2596 to track improving the usability of baselines with arguments.