Feature request: Support combining benchmark results
Opened this issue · 0 comments
The following use case happened often enough to trigger me to write this feature request: I often want to benchmark a modification made to the system, but this modification would need some effort to script. So I would like to call hyperfine, then modify the system, then call hyperfine again. Instead of having the result of two individual results, I like to have a nice comparison of those two.
Recent example: I wanted to benchmark the performance impact of the patch submitted in dracut-ng/dracut-ng#1678. So I created a chroot with Dracut installed. Then I ran hyperfine "dracut --debug --force" for the baseline result. Then I applied the submitted patch to /usr/lib/dracut/dracut-logger.sh and ran hyperfine "dracut --debug --force" again. Now I have those results and have to calculate the difference myself:
# hyperfine "dracut --debug --force"
Benchmark 1: dracut --debug --force
Time (mean ± σ): 8.427 s ± 0.145 s [User: 12.157 s, System: 4.686 s]
Range (min … max): 8.208 s … 8.745 s 10 runs
# hyperfine "dracut --debug --force"
Benchmark 1: dracut --debug --force
Time (mean ± σ): 8.306 s ± 0.071 s [User: 11.887 s, System: 4.624 s]
Range (min … max): 8.220 s … 8.426 s 10 runs