SPHINCS+ SHAKE aarch64 implementation is never enabled

Discussed in https://github.com/orgs/open-quantum-safe/discussions/1965

^{Originally posted by wadaphaq October 28, 2024}
When testing the algorithms' performace with and without CPU extensions, I noticed that sphincs shake algorithms have same performance with or without the CPU extension enabled in the building process. Upon inspection of the liboqs I noticed that the cmakelists in the sphincs src folder do not include an aarch64 version, and even after building liboqs the aarch64 version is missing from the source folder. Is there a way to activate and build the aarch64 version of the sphincs shake algorithms?

It looks to me like the aarch64 implementations of SPHINCS+ were added to the library in PR #1420, but they are present in an "ignore" list here:

liboqs/scripts/copy_from_upstream/copy_from_upstream.yml

Line 24 in 7132473

    
           ignore: pqclean_sphincs-shake-256s-simple_aarch64, pqclean_sphincs-shake-256s-simple_aarch64, pqclean_sphincs-shake-256f-simple_aarch64, pqclean_sphincs-shake-192s-simple_aarch64, pqclean_sphincs-shake-192f-simple_aarch64, pqclean_sphincs-shake-128s-simple_aarch64, pqclean_sphincs-shake-128f-simple_aarch64, pqclean_kyber512_aarch64, pqclean_kyber1024_aarch64, pqclean_kyber768_aarch64, pqclean_dilithium2_aarch64, pqclean_dilithium3_aarch64, pqclean_dilithium5_aarch64

This means that the aarch64 directories are not updated with the copy_from_upstream script. In fact, if you delete the aarch64 implementations and then rerun copy_from_upstream, the deleted directories are not replaced.

@dstebila @baentsch Do you happen to remember what the intent was with regard to these schemes in #1420? Did we want to exclude them from the library for some reason? (This was before my time.)

@dstebila @baentsch Do you happen to remember what the intent was with regard to these schemes in #1420? Did we want to exclude them from the library for some reason? (This was before my time.)

I don't recall for sure, but I think it was just that we only made the effort for the generic implementation at the time.

@dstebila @baentsch Do you happen to remember what the intent was with regard to these schemes in #1420? Did we want to exclude them from the library for some reason? (This was before my time.)

I don't recall for sure, but I think it was just that we only made the effort for the generic implementation at the time.

I don't recall either -- but do remember that Sphincs never was one of our "favourite" algorithms, so may very well have been treated along the lines of "just make it work", i.e., "skip optimizations if they cause headaches" -- particularly right before/potentially delaying a release.

After looking into this, I don't think this will be as simple as simply "switching on" the aarch64 implementations—work would need to be done on FIPS 202 shims (for which we would need "x2" versions in addition to the exsting "x4 versions).

I suspect the aarch64 code was pulled in via a copy_from_upstream run on an earlier commit on the dev branch for #1420 before the aarch64 implementations were added to the ignore list. copy_from_upstream would never have deleted the code afterward, and it was presumably missed in code review due to the ridiculous number of files changed in the src/sig/sphincs directory. We don't advertise an aarch64 implementation for SPHINCS+ anywhere in our documentation.

I would be inclined to simply delete the aarch64 directories for now and deal with aarch64 implementations as part of #1894 (if we want to support ARM-optimized implementations of SLH-DSA). It would seem to be a waste of effort to have to redo the integration work to accommodate further upstream changes.