Tuning the benchmark repeat rate.

[ben-marshall]

Hi there

Would it be possible to add a switch / knob somewhere in the configure script to tune the repeat rate for each benchmark?

Currently, the number of repetitions is a function of the global CPU_MHZ define, and a local (to the benchmark) LOCAL_SCALE_FACTOR (magic?) number. It would be nice to be able to change the LOCAL_SCALE_FACTOR to something very small for testing, or running simulations of the hardware where sometimes just 10's of repetitions suffices to get an idea of performance.

This would be very useful when analysing hardware being developed to really drill down into why performance bugs occur. Typically one would want access to waveform dumps, which grow too big to handle very quickly. Only running a few iterations keeps the waveform dumps small enough to use.

I'm happy putting together a patch for this myself, but figured I'd ask here first.

Cheers,
Ben

[jeremybennett]

Hi Ben,

Thanks for the suggestion.

There is a danger in making benchmarks too tunable. You want relatively few knobs, and those knobs very well defined to ensure results are comparable.

Since the benchmarks are open source, there is nothing stopping you hacking the code to change LOCAL_SCALE_FACTOR. If you are a developer looking at wave traces, this will be easily within your skills.

Possibly a discussion to have on the mailing list (see https://lists.librecores.org/listinfo/embench), so I'll close the issue here. Feel free to reopen, or open a new issue if the mailing list discussion comes to a conclusion.