smarr/ReBench

[T.1.2] Expanding Machine Configuration and Validation.

smarr opened this issue · 0 comments

smarr commented

ReBench already configures benchmarking machines to minimize measurement error. The [Krun benchmark runner](https://github.com/softdevteam/krun} supports additional configuration settings and operating systems. This includes active thermal management, additional setting validation, and perturbation detection.

Additionally, we currently do no have any active setup validation based on basic and well understood benchmarks, which would be a recommended best practice.

For instance for a 20min benchmark job, it would be sensible to have a quick check at the beginning to see whether basic expectations of the machine can be replicated.

It would significantly increases confidence into the benchmarking setup.