Add benchmarking code

Question

Add benchmarking code

Closed this issue 3 months ago · 4 comments

I'd like to have benchmarking code here for at least the following 3 cases

PMU cycles for platforms
PERF performance for systems where access to the PMU cycles counters is not possible, but perf is compiled into the Kernel.
APPLE for Apple Mx cores based on https://gist.github.com/dougallj/5bafb113492047c865c0c8cfbc930155#file-m1_robsize-c-L390

https://github.com/slothy-optimizer/pqax has all that and is licensed under MIT which means we can take some of that code.

Am I missing something?

At some point it would be nice to have automatic benchmarking on a number of platforms at some point.

Answer 1 · 2024-03-27T16:14:46.000Z

Good idea. Github Action supports Apple M series just recently. Other CPUs are run under QEMU.
Other than Apple M series, Circle CI and Github Action don't specify the actual CPU, my quick 5-min search show the options are: armv7 or aarch64.

I would vote for the Apple counter, and PERF for other ARM CPUs so it's can be as generic as possible.

Answer 2 · 2024-03-27T16:24:16.000Z

I doubt that we can benchmark on the Github runners. But we can host our own runners eventually.

I wasn't aware of the M1-based runners. Testing on those would be nice. It looks like they are not free to run, right? Maybe we keep that in mind, but first having qemu-based testing is sufficient I think.

Answer 3 · 2024-05-10T04:38:37.000Z

Let's separate the issue of provisioning the benchmarking code from provisioning benchmarking automation / CI.

Answer 4 · 2024-06-28T07:09:52.000Z

Completed in #81