Access to ARM github runners
Closed this issue · 4 comments
Currently the open-quantum-safe team is using circleci to build arm code.
Github made ARM64 runners available this year. Though initially private beta (Feb-Jun), our organization has access . However we don't currently have funding for the usage.
Related:
- build issue in liboqs - where arm runners would help
- Request to get funding for ARM runners at oqs tsc
- Github private beta repo (restricted access)
@mkannwischer @hanno-becker Would this be useful for our arm repos?
If so we might be able to estimate usage & see if we can also raise with the PQCA TAC -- which has the option of requesting funding from the Governing board.
Thanks for bringing this up @planetf1.
Testing on github's AArch64 runners sounds useful in general, but I'd assume we cannot reliably benchmark on there.
For the mlkem-c-aarch64, we will require benchmarking (at least on the platforms we are targeting) and I hope that we can get that done by CI as well (see pq-code-package/mlkem-c-aarch64#34). If we have that up there is little to no benefit running on Github's AArch64 runners as well.
So unless it's low effort and negligible cost, I don't think mlkem-c-aarch64 or mlkem-c-embedded need access to the Github AArch64 runners for now.
We do have some experience with emulating AArch64 using qemu running on the default Github runners: https://github.com/slothy-optimizer/pqax/blob/main/.github/workflows/tests.yaml. That is free, but of course if there is a choice running it natively would be preferred.
@mkannwischer Thanks for the reply. I would hope the effort and costs are low, and would give similar results to your existing qemu tests, except easier. But for benchmarking it seems as if dedicated hw is needed (either custom integration - building a pool of custom github runners possible too - but my concern would be overhead on benchmarking systems?).
Perhaps this is one to monitor, to see how the oqs team progress. That will help clarify costs, and in July the capability should be GA in any case.
I agree with @mkannwischer that we don't need Arm runners for functional testing -- qemu is good enough. We do need access to specific hardware (see the README) for benchmarking, but it appears that github's AArch64 runners would not give that flexibility.
As per last TSC on 20240620, I think we can close this now @hanno-becker as #75 will continue the followup to get AWS environment for benchmarking?