BenchSweeps.jl wraps BenchmarkTools.jl to:
- Setup grid of parameters and define benchmarks for all combinations
of them. Those parameters and defined benchmarks are bundled
together by
BenchSweepGroup
. - Access benchmark data using dataframe/table-like interface.
BenchSweepGroup
supports Tables.jl and IterableTables.jl API and more controllable interface for DataFrames.jl. - Save/load
BenchSweepGroup
to/from a JSON file.
using BenchSweeps
using LinearAlgebra
suite = BenchSweepGroup()
suite.axes[:n] = 2 .^ (2:5)
suite.axes[:m] = 2 .^ (2:5)
@defsweep! for (n, m) in suite["matrix-matrix"]
@benchmarkable mul!(Y, A, X) setup=begin
Y = zeros($n, $m)
A = rand($n, $n)
X = rand($n, $m)
end
end
results = run(suite)
using DataFrames
df = DataFrame(results) # now analyze the benchmark