fail_on_failure = false marks the step as failed when a test fails

[fasmat]

I have a job that runs unit tests. Some of these tests fail. I have the following step in my job:

- name: Publish Test Report
  uses: mikepenz/action-junit-report@v3
  if: always()
  with:
    report_paths: "**/unit-tests.xml"
    fail_on_failure: false

This step fails when any test in unit-tests.xml failed, although fail_on_failure is set to false.

[mikepenz]

@fasmat do you have some additional details on this please.

The action fails either if:

• token missing https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L12-L15
• no test cases are found https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L75-L80
• there is a failure and the flag is true https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L124-L126
• there was an exception https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L129-L131

It would be helpful to see with what message it fails.

[fasmat]

Hi @mikepenz, thanks for your quick response.

A token is set, and tests cases were found. When all tests pass the build step is marked as successful and shows the number of executed and skipped tests.

If some tests fail the step executing the tests is marked as successful, but the step using mikepenz/action-junit-report@v3 using fail_on_failure: false is marked as failing:

[fasmat]

The full run can be found here: https://github.com/spacemeshos/go-spacemesh/actions/runs/3183085385
and the relevant lines in the pipeline configuration are here: https://github.com/spacemeshos/go-spacemesh/blob/c814f43c67b66a260f979aa58097e4909b4f0abc/.github/workflows/ci.yml#L162-L174

[mikepenz]

Oh I see now. It's not the step which fails:

You are referencing to the status of the check:
https://github.com/mikepenz/action-junit-report/blob/main/src/annotator.ts#L96

Currently that one will have the respective conclusion of the test results.

If you only want the annotation you could enable annotate_only=true and have a job summary being posted at the end

[fasmat]

annotate_only=true did the trick. Thanks 😃