bytecodealliance/sightglass

sightglass-next: implement sightglass-server

abrown opened this issue · 2 comments

In order to report performance results based on PRs, we talked about implementing an HTTP server (e.g. in crates/server) that would:

  • listen for incoming POST requests that contain JSON with the PR URL, commit SHA, etc. necessary for doing a "master vs PR" comparison
  • to avoid DoS, verify that the request is an authorized one (not exactly sure how to do this but the GitHub action will need some form of token)
  • kick off some benchmark running, like sightglass-cli benchmark ... but we could call the same APIs from inside the web service
  • upon success, push a Markdown table of the significant performance differences as a comment to the GitHub PR (implies that sightglass-server has a GitHub token, like one from bytecodealliance-highfive)
  • upon failure, push the error message as a comment to the GitHub PR (implies that we maintain error logs somewhere)

There is a lot more to be done here but that should be a workable start.

@abrown thanks for posting this. Finally getting to here and wanted to acknowledge this. Before doing this particular issue will first refactor work on the original sightglass. That hopefully won't take long.

kick off some benchmark running, like sightglass-cli benchmark ... but we could call the same APIs from inside the web service

I think we actually do want to spawn the CLI in a subprocess because all of the random ordering of processes/iterations is inside the CLI crate.