Move away from comparative speed method names
nirvdrum opened this issue · 2 comments
This is a proposal to move away from method names like slow, fast, and fastest and instead use something named after the operation. The convention is a bit error prone when authoring the benchmark to begin with, since you can't know a priori what the relative speeds are until you've written the benchmarks. Thus, it requires a renaming of the methods after the benchmark is run. It may introduce confirmation bias as well.
The real problem, I think, is it risks the entire repository becoming bit rotted. Ruby's performance is not fixed in time. If Ruby 3 manages to hit its 3x performance gain, it may very well come at the cost of reshuffling the relative performance of methods involved in these benchmarks. Then we'd be looking at the unenviable case of the fast method being faster than the fastest method and so on.
Going along with performance being implementation-specific across MRI versions, the problem is exacerbated when running the benchmarks on alternative Ruby implementations. In that case, the relative method values do not hold in many cases.
I don't think anyone would be against a PR that updated/changed all of the current benchmarks in the way that you propose. 👍
Sounds good, we can continue the discussion in #215