python/pyperformance

Include an overall summary number in compare output

itamaro opened this issue · 2 comments

itamaro commented

People often quote a single perf number when comparing versions (e.g. "Python 3.11 is 25% faster than Python 3.10")

The pyperformance compare command prints out the "change factor" per benchmark. It's not clear how to go from this list of numbers to a single overall number. Having the compare command compute and print it out would also help with consistency when quoting this single perf number (is it mean? geometric mean? only including significant results or all of them?)

itamaro commented

ok, just learned this can be done with pyperf

pyperf compare_to baseline.json change.json
ericsnowcurrently commented

It would be worth updating pyperformance though.

  • 👍1