Failed tests are not shown in Annotations

[GitHubVadym]

Hi,
I'm using such step in gh actions:

      - name: Publish Test Report
        uses: mikepenz/action-junit-report@v3.7.8
        if: success() || failure() # always run even if the previous step fails
        with:
          report_paths: '**/build/test-results/test/TEST-*.xml'

In the end, I see such a report, but failed test(assertion) is not shown in the annotations.

Am I doing something wrong or is it a bug?

[mikepenz]

By default the action will create the annotations via the checks API of GitHub: https://github.com/mikepenz/action-junit-report/blob/main/src/annotator.ts#L85-L102

Would you like to see annotations within the test files? or your aim is to have the report of files which failed in the summary.

For the later you want to enable this by setting detailed_summary to true

[GitHubVadym]

Unfortunately, there are no annotations created in my Job Summary like in this example. https://github.com/mikepenz/action-junit-report/blob/main/.github/images/annotations.png
As you may see at my first screen I have only 1 annotation that build failed

[mikepenz]

I would need to see more logs for that, possibly with a sample to try to see why that may be the case.

An example build where such annotations are generated from this repos tests: https://github.com/mikepenz/action-junit-report/actions/runs/5304280643

And this is from the samples with a configuration like: https://github.com/mikepenz/action-junit-report/blob/main/.github/workflows/build.yml#L18-L25

[GitHubVadym]

@mikepenz
I think there is a problem when workflow is triggered by workflow_dispatch:
When I run wf from github UI or rest call I get no failed tests in anotations:

When workflow is run on call, or on push: I see failed tests in anotations:

In my configuration I have 2 workflows where first is calling second:
First wf:

---
name: Tests workflow caller

on:
  push:
    branches: [ main ]
  workflow_dispatch:
  workflow_call:
  schedule:
    - cron: '13 */1 * * *'
  pull_request:
    branches: [ main ]
    paths: "**/tests_caller.yml"

permissions:
  actions: read
  checks: write
  contents: write
  issues: read
  packages: write
  pull-requests: write
  statuses: read
  security-events: write
  id-token: write

concurrency:
  group: ${{ github.head_ref }}-${{ github.workflow }}-${{ github.event_name }}-dev
  cancel-in-progress: true

jobs:
  dev-api-e2e-tests:
    name: run tests workflow
    uses: ./.github/workflows/tests.yml
    with:
      environment: dev
    secrets: inherit

Second workflow where actual tests are running:

---
name: Test

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment'
        type: string
        default: dev
        required: true
  workflow_call:
    inputs:
      environment:
        description: 'Environment'
        type: string
        required: true

permissions:
  contents: read
  checks: write
#  id-token: write

jobs:
  runIntegrationTests:
    name: e2e tests
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - name: "Checkout Code"
        uses: actions/checkout@v3

      - name: Setup JDK
        uses: actions/setup-java@v3
        with:
          distribution: temurin
          java-version: 17

      - name: "Make gradlew executable"
        run: |
          chmod +x ./gradlew

      - name: Run Integration Test
        run: ./gradlew test --info

      - name: Publish Test Report
        uses: mikepenz/action-junit-report@v3.7.8
        if: success() || failure() # always run even if the previous step fails
        with:
          check_name: JUnit Test Report
          report_paths: '**/build/test-results/test/TEST-*.xml'
          summary: '<table><thead><tr><th> Application (src/applications) </th></tr></thead><tbody><tr><td> test </td></tr></tbody></table>'
          check_title_template: '{{SUITE_NAME}} | {{TEST_NAME}}'
          detailed_summary: true

      - name: Some extra step
        run: echo "Test finished on ${{ inputs.environment }}"

[GitHubVadym]

same issue with run on schedule

[mikepenz]

Ok sorry for the late answer. Tested it now here: https://github.com/mikepenz/action-junit-report/actions/runs/5552550460

And it actually makes sense. Given there is no PR - so the action has no place where it can actually create the check for. Like how it would do here: https://github.com/mikepenz/action-junit-report/actions/runs/5552522553

So for this particular usecase I'd suggest to use annotate_only.

Which will result in an output like this: https://github.com/mikepenz/action-junit-report/actions/runs/5552618163

[mikepenz]

(E.g. 0a5b1eb)

[GitHubVadym]

Thank you @mikepenz
annotate_only: ${{ github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'}} - works in my case

[mikepenz]

That's great to hear!