Doesn't include errors

[sheikhasim]

Junit report shows 0 failures and 11 errors.
And i expect the action to detect errors as well and fail using fail_on_failure.
But it shows all passing.

testsuites tests="25" failures="0" errors="11" time="17.361686"

ℹ️ - JUnit Test Report - 25 tests run, 25 passed, 0 skipped, 0 failed.
ℹ️ - JUnit Test Report - Creating check for

[mikepenz]

Seems like the report is not a valid junit export.

First of it seems it reports errors instead of failures, furthermore the action does not use those test suite level values, but instead it counts the actual failures as reported in the testcases:

Some examples:

• https://github.com/mikepenz/action-junit-report/blob/main/test_results/cunit/testFailure.xml#L9
• https://github.com/mikepenz/action-junit-report/blob/main/test_results/xunit/report.xml
• https://github.com/mikepenz/action-junit-report/blob/main/test_results/issues/testFailedDisabled.xml

[sheikhasim]

Ohh , found a workaround for now for detecting errors in test reports.
How about having an option to fail_on_detected_errors as well ?

[mikepenz]

Thinking about that, I am not certain this is the best choice. As far as I can tell based on the information of this ticket, the provided junit export does not fully meet the format requirements, including the testcases and their failures, which is the main reason it didn't report the problem.

Which framework for exporting the test case do you use? or are those reports manually generated?