Test report stopped working without change
marcheyer opened this issue · 8 comments
Hello,
we use this plugin for quite a while and it worked perfectly.
Since yesterday our builds fail because it creates an error.
This was the output for the last working task:
Run mikepenz/action-junit-report@v3
with:
report_paths: source/app/build/logs/checkstyle.xml
fail_on_failure: true
token: ***
annotate_only: false
update_check: false
check_name: JUnit Test Report
require_tests: false
include_passed: false
check_retries: false
transformers: []
job_summary: true
detailed_summary: false
annotate_notice: false
follow_symlink: false
job_name: php_codestyle
env:
COMMIT_USER: Github
COMMIT_EMAIL: @.***
DOCKER_REGISTRY: ***
DOCKER_IMAGE_BASE: ***
📘 Reading input values
📦 Process test results
Retrieved 1 reports to process
Now it changed to:
Run mikepenz/action-junit-report@v3
with:
report_paths: source/app/build/logs/checkstyle.xml
fail_on_failure: true
token: ***
annotate_only: false
update_check: false
check_name: JUnit Test Report
require_tests: false
include_passed: false
check_retries: false
transformers: []
job_summary: true
detailed_summary: false
annotate_notice: false
follow_symlink: false
job_name: php_codestyle
env:
COMMIT_USER: Github
COMMIT_EMAIL: @.***
DOCKER_REGISTRY: ***
DOCKER_IMAGE_BASE: ***
📘 Reading input values
📦 Process test results
Retrieved 1 reports to process.
ℹ️ Posting with conclusion 'failure' to refs/heads/stage (sha: 683280882cda63bb975ca970006772d8c861e4e5)
🚀 Publish results
ℹ️ - JUnit Test Report - No test results found!
ℹ️ - JUnit Test Report - Creating check for
Error: ❌ Tests reported 0 failures
Can you explain why the behavior changed? The checkstyle.xml artifact is in both cases the same:
@marcheyer you are referencing to the @v3 tag - which will get all v3 updates.
Yesterday there was a bugfix release which fixed a faulty condition breaking out too early: https://github.com/mikepenz/action-junit-report/pull/797/files#diff-4fab5baaca5c14d2de62d8d2fceef376ddddcc8e9509d86cfa5643f51b89ce3d
Looks like your code would originally also run into the return condition found here:
https://github.com/mikepenz/action-junit-report/blob/v3.7.2/src/main.ts#L88
Which was now changed to:
https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L90-L93
Meaning given that you defined failOnError = true with a failure conclusion, it fails now here:
https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L129-L131
If you do not want fixes to automatically affect your builds, it is always highly encouraged that you either reference to the full version.
Which means in your case you would use:
mikepenz/action-junit-report@v3.7.2
Or even better, it is advised from security perspective to pin actions from 3rd party to the full git hash, which means no modification ever is possible to the action you use. (theoretically git actions maintainers still could move the v3.7.2 tag modifying your behaviour (which we do not do) - just noting for full clarity)
Resulting in
mikepenz/action-junit-report@f89327e8361cf2a68997a18fcc3e2a21b4991f2d
Anyways. I'd suggest to use the new update as it is meant to fix a bug.
Hey @mikepenz,
at first thanks for your fast reply!
I don't understand why the test is counted as failed. The output itself indicates there are 0 errors.
Can you point this out for me?
Based on the code it would mean the totalCount resulted in 0: https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L98
Which matches with JUnit Test Report - No test results found!
I suppose given we fixed: https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L91-L93
We want to change this condition to only care about failures:
https://github.com/mikepenz/action-junit-report/blob/main/src/main.ts#L98
I see. Then i have to use the fixed version since i can't easily change the output of this test.
Thank you very much for the fast Support!
There will be an update in a bit.
@marcheyer please retry the run.
This works for us. Thank you very much