anp/lolbench

Use measureme for instruction counts

anp opened this issue · 0 comments

anp commented

One of lolbench's main tricks is to use hardware performance counters in addition to wall time for detecting anomalies. I first implemented this using a fork of criterion which in turn makes use of a very MVP interface I wrote against perf_event_open.

Since then, https://lib.rs/crates/perf-event-open-sys was published and has apparently gotten quite mature!

@eddyb recently posted a fantastic PR to measureme adding support for instruction counts making use of the perf-event-open-sys crate, and has reported some very low-noise results using it for rustc benchmarks.

There are a few places to possibly get started, depending on ambition levels:

  1. move my mega-old criterion fork to use measureme's counter
  2. make a newer criterion fork that uses measureme's counter
  3. move off of criterion and define our own bench runner which uses measureme's counters

(1) seems easiest, (3) is what I'd actually like to do given the difficulty of doing sound statistics with the summary stats reported by criterion. But that might be too ambitious for an initial outing.