Adopt a definition of assurance levels
Opened this issue · 5 comments
I'm interest in assisting with this effort when it gets started.
@TheFoxAtWork At the PQCP TSC last week we discussed this topic briefly (1st, so lots to get through), and agreed that a good starting point would be for the experts in each subproject to share their perspective, and that we'd then try to normalize across the subprojects.
From my perspective we need to be able to articulate clearly what a consumer should expect from each algorithm, and help them understand how to decide.
Matthias is going to share the views from his projects mlkem-c-embedded and mlkem-c-aarch64at the tsc next week
Thank you for the update!
I would suggest a few labels with different assurance levels that we can add to repositories, and a document that defines them. On top, we should have additional, more fine-grained properties each library defines, but they are unlikely to be understood by many consumers.
High level it could be something like
- AL 1 (formally verified | where this has a definition that we must agree on, probably something like functional correctness)
- AL 2 (exhaustively tested | where this has a definition that we must agree on, probably something like property based testing, kats, fuzzing, etc.)
- AL 3 (tested | self tests, kats)
The more fine-grained properties may then be something like
- secret independence (machine code, or before compilation)
- more properties ...
A label for "audited" may also be nice to have.
I've tried to harvest the good ideas above and capture in an initial doc page.
- pq-code-package/documentation#8 has a very early rough page - we could discuss at TSC. Based on a simple list not a taxonomy/rich ontology. None of the definitions are rich, nor are the levels correct.
- OpenSSF has various checklists (scorecard, cii initiative, a base level for open source best practice) but so far none go to this kind of level. Asking in this community to see if any related activities