Show 🎄
qstokkink opened this issue · 4 comments
We have had extended unit tests run on validate since 4cf28ad, lovingly referred to as the "Christmas tree". 🎄
Initially, these tests were not added as a PR check, pending assessment of their stability.
By now, these tests are stable enough to detect issues (e.g., in #1257). However, they are not shown as a PR check, so the test results are hidden for the PR author.
We should modify .github/workflows/pr-comment-validate.yml to report its status on the corresponding PR.
After a lot of digging (through repos and tutorials), I found https://github.com/marketplace/actions/github-status-action-v2 to set commit statuses. This action seems to (1) still be maintained and (2) able to set statuses based on our triggering event, which is:
py-ipv8/.github/workflows/pr-comment-validate.yml
Lines 3 to 5 in d2d31ee
However, the issue_comment event does not include the commit SHA to set the status for. Open problem: we have to somehow fetch that from the issue number.
Our current approach for the checkout action is to create a remote ref from the issue number. We should - probably - be able to dereference this (in a preceding step) and feed it to github-status-action-v2.
I can't find any action to grab a SHA from a PR ref. We'll probably need a custom command and write it to an env var.
The command itself should be something like the following:
git ls-remote origin refs/pull/${{ github.event.issue.number }}/head | grep -o "^\w*\b"
The origin may also need to be determined from the issue_comment. We can try to see if it works without that.
Whatever it may end up looking like: we can then set the env var and feed it to github-status-action-v2. This should be enough info to start the implementation.
Lesson learned: env vars can only be used within the steps of a single job. To communicate output of jobs between jobs we need to write to $GITHUB_OUTPUT instead of $GITHUB_ENV (https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs).
🎉 The results are now visible in the PR, this issue is resolved.
