dorny/test-reporter

Test report doesn't appear in expected place in GitHub UI

tsimbalar opened this issue · 15 comments

tsimbalar commented

Hi,

thanks a lot for this Action 👍

We currently have multiple GitHub Actions Workflows (main.yaml , lint-yaml.yaml etc ).

When using the action inside of the main.yaml, the test results appear under the lint-yaml results 🤔

Not 100% sure what is causing it

image

our step, within main.yaml looks like

    - name: Report PR Test results
      uses: dorny/test-reporter@v1
      if: ${{ github.event_name == 'pull_request' && (success() || failure()) }}
      with:
        name: jest Tests
        path: reports/jest-*.xml
        reporter: jest-junit

Here is a (sanitized) output of the step :

 1s
Run dorny/test-reporter@v1
  with:
    name: jest Tests
    path: reports/jest-*.xml
    reporter: jest-junit
    list-suites: all
    list-tests: all
    max-annotations: 10
    fail-on-error: true
    token: ***
  env:
     // snipped
Action was triggered by pull_request: using SHA from head of source branch
Check runs will be created with SHA=8d1d74409581269df26b4df491b55819597d9501
Listing all files tracked by git
  /usr/bin/git ls-files -z
  // ...
Found 190 files tracked by GitHub
Using test report parser 'jest-junit'
Creating test report jest Tests
  Processing test results from reports/jest-junit.xml
  Creating report summary
  Generating check run summary
  Creating annotations
  Creating check run with conclusion success
  Check run create response: 201
  Check run URL: https://api.github.com/repos/XXXX/YYYY/check-runs/2005347522
  Check run HTML: https://github.com/XXXX/YYYY/runs/2005347522
tsimbalar commented

Oh, I see.

Thanks for the reply 👍

dorny commented

Unfortunately as @lhotari already commented - it's a GitHub "feature".
Another post on this is here: https://github.community/t/specify-check-suite-when-creating-a-checkrun/118380

Until it's fixed on GitHub side, the best I can do is to mention it in README.

  • 👍1
skrysmanski commented

Just a quick thought (and sorry if this has been discussed before):

Instead of creating a check, couldn't you use the same mechanism that https://github.com/actions/upload-artifact uses (under the assumption that you can upload other file types than .zip files)?

This would solve the problem in this issue. It would also solve the problem that this action requires the checks: write permission (which is not available for pull requests from forks).

I understand that this change would (most likely) worsen the UX of this action - but on the other hand, I'd argue that you don't need to view the test report that often (usually only if some test has failed), so this change could be ok.

  • 👍1
itungnt commented

This also happen when you have a schedule trigger workflow.
The test report is not attached to each of the run.

  • 👍1
kuhnroyal commented

Maybe the new job summary (https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/) can be used to:

  1. display they success/fail count like in the blog post
  2. display link to the check run in order to easier find it?
  • 👍26
nipunn1313 commented

Hi! Thanks for the work on the reporter!
Tiny UX request regarding this issue

Would it be possible to move the URL link out of the dropdown?
image

(Here's what it looks like currently with the dropdown)
image

Given that the check appears in an unexpected place on a different workflow (due to the preexisting github shortcoming) - it would be nice to have more obvious breadcrumbs (the HTML link) visible without having to open the arrow - to help new folks find the errors.

flobernd commented

I see that there is a PR to add the "job summary" feature to an unofficial fork of this action: phoenix-actions/test-reporting#21

plewkowycz commented

The same issue for me

djbrown commented

+1 for option to publish as job summary, instead of check

djodjo02130 commented

+1 for option to publish as job summary, instead of check

rr-jino-jose commented

can we get the 'Check run HTML' URL as output of this report??

  • 👍1
JonathanAtCenterEdge commented

+1 for option to publish as job summary, instead of check

  • 👍8
huehnerlady commented

+1 for option to publish as job summary, instead of check

timcassell commented

Job summaries are already used on main. We just need a new release.

  • ❤️2