mikepenz/action-junit-report

Action is slow - with `include_passed: true`

vitalyk-multinarity opened this issue · 11 comments

I'm using this action for publishing JUnit report generated by 'ctest'.
JUnit report with 176 tests, from them 84 skipped; and it takes about 35 minutes, mostly in "Process test results" step.
Is there a way to improve action performance?
Thanks, Vitaly

@vitalyk-multinarity 35 minutes? That seems intense.

Which configuration do you use?


Given that annotateOnly has to iterate over all annotations, and uses the GitHub interface to annotate each and all of them: https://github.com/mikepenz/action-junit-report/blob/main/src/annotator.ts#L31-L45 (with annotateNotice) it wouldn't surprise me to take longer.
As far as I am aware there is no batch API for these available.

Ways to speed up in that case:

  • annotateNotice = false
  • use the checks instead of annotate only

I use this action with thousands of test-cases in different projects and it usually completes within seconds.
For these cases we do have annotateOnly = false and use the normal checks + summary.

@mikepenz - thank you!
I re-read the Readme, but I'm still not sure what is effect of "annotate_notice = true/false".
My configuration is

uses: mikepenz/action-junit-report@v3
        if: always()
        with:
          report_paths: '/tmp/ctest-*-report.xml' 
          fail_on_failure: true
          detailed_summary: true    
          include_passed: true 

which converts to:

Run mikepenz/action-junit-report@v3
  with:
    report_paths: /tmp/ctest-*-report.xml
    fail_on_failure: true
    detailed_summary: true
    include_passed: true
    token: ***
    exclude_sources: /build/,/__pycache__/
    annotate_only: false
    update_check: false
    check_name: JUnit Test Report
    require_tests: false
    check_retries: false
    transformers: []
    job_summary: true
    annotate_notice: true

@vitalyk-multinarity ok given the output you are not using annotate_only: true so that's not the source of the problem.

Given the other issue you had created I understand that you will want to use include_passed: true, but would it possible to compare speed with it disabled?

Without "include_passed: true" it takes just 2 seconds :-)

@vitalyk-multinarity if I remember correctly. you use include_passed mainly to get them also included in the summary. but you actually don't mind the annotations at the specific file locations. correct?

If so, I'll look into it and check optimisations we can apply to skip any file matching and similar.

if I remember correctly. you use include_passed mainly to get them also included in the summary.
Right (to be precise, to see a list of passed tests with their status)!

If so, I'll look into it and check optimisations we can apply to skip any file matching and similar
Many thanks!

Could you please test how it performs using f4441d735cac5cebc8f7c6dadf8eef97865b4d32?

See the related PR here: #658

  • ultimately disabling the resolving of the path on the filesystem unless annotation for passed tests is requested.

(still need to adjust tests for this PR due to the adjusted handling for the annotations)

Many thanks - with this commit it takes just one second, as far as I added 'annotate_notice: false':

 uses: mikepenz/action-junit-report@f4441d735cac5cebc8f7c6dadf8eef97865b4d32
 if: always()
 with:
       report_paths: '/tmp/ctest-*-report.xml' 
       fail_on_failure: true
       detailed_summary: true    
       include_passed: true
       annotate_notice: false 

Thank you for testing this.

This will be in the next release, which will probably be released later this week.

You can use the git commit SHA pinned version until then

Mike, many thanks!
Vitaly