psf/pyperf

Output geometric mean to 0.1% precision

Opened this issue · 3 comments

Outputs such as "0% slower" or "1% faster" are unnecessarily vague.
Is "1%" 0.5% or 1.5%?

Do you suggest to add one more digit in the output? Do you want to propose a PR for that?

While 0.4% on individual benchmarks is probably not noteworthy, it's nice to see on the geometric mean of an entire suite. Very few changes will ever move the pyperformance geometric mean more than a couple of percentage points, so most of the time we see one of (1.01 slower, 1.00 slower, 1.00 faster, 1.01 faster, 1.02x faster), which isn't very much information IMO.

Another option would be to add three decimal places to pyperformance only (which doesn't display geometric mean at all yet).

While 0.4% on individual benchmarks is probably not noteworthy, it's nice to see on the geometric mean of an entire suite.

+1