jonhoo/rust-ci-conf

Suggestion: add in automated benchmarking

jembishop opened this issue · 2 comments

I was watching your most recent Rust Nation talk and you mentioned automated benchmarking to detect regressions. I was wondering if you had any CI configs that you could upload to this repo, if you could be so kind? If not maybe something based on what they have in rusttls would be a good idea.

Unfortunately I think benchmarking is, at the moment, too "up in the air" to standardize in here. I suppose we could have files that check for the existence of things like benches/tango.rs or benches/iai.rs, but feels unnecessary to waste a bunch of CI resources on spinning up a job just to then not execute because those files don't exist.

Something I've done recently instead of merging this repo into my own repos is to setup a set of CI templates which I include into each repo by calling the workflows. https://github.com/joshka/github-workflows/
Using that approach, setting up a couple of standard benchmark templates that can be easily called might be a viable approach for this - similarly this could be a reasonable approach for the other more optional CI workflows in this repo.