seriyps/rebar3_bench

Add "compare" command

Opened this issue · 0 comments

Would be nice to be able to not only runn benchmarks and compare results with baseline, but also to be able to define 2 implementations at the same time and compare them. Eg

-module(my_kv).
-export(...).

maps_update(input) ->
    maps:from_list([{K, K} || K <- lists:seq(1, 5000)]).

bench_maps_update(Map) ->
    Map#{2500 => -1}.

dict_update(input) ->
    dict:from_list([{K, K} || K <- lists:seq(1, 5000)]).

bench_dict_update(Dict) ->
    dict:store(2500, -1, Dict).
$ rebar3 bench cmp my_kv:maps_update my_kv:dict_update

And this command will run 2 benchmarks and do all the statistical computations and comparision and will tell which implementation is faster.