/BenchSweeps.jl

Primary LanguageJuliaMIT LicenseMIT

BenchSweeps

Build Status Codecov Coveralls

BenchSweeps.jl wraps BenchmarkTools.jl to:

  • Setup grid of parameters and define benchmarks for all combinations of them. Those parameters and defined benchmarks are bundled together by BenchSweepGroup.
  • Access benchmark data using dataframe/table-like interface. BenchSweepGroup supports Tables.jl and IterableTables.jl API and more controllable interface for DataFrames.jl.
  • Save/load BenchSweepGroup to/from a JSON file.

Usage:

using BenchSweeps
using LinearAlgebra

suite = BenchSweepGroup()
suite.axes[:n] = 2 .^ (2:5)
suite.axes[:m] = 2 .^ (2:5)

@defsweep! for (n, m) in suite["matrix-matrix"]
    @benchmarkable mul!(Y, A, X) setup=begin
        Y = zeros($n, $m)
        A = rand($n, $n)
        X = rand($n, $m)
    end
end

results = run(suite)

using DataFrames
df = DataFrame(results)  # now analyze the benchmark