unable to import results running cypress
j-morin opened this issue · 6 comments
I have been having trouble exporting my test results into xray successfully from github actions using cypress. Here are the results I am seeing (see attached file).
Here is how I have it set up in my yml file
- name: Submit results to Xray
uses: mikepenz/xray-action@v2
with:
username: ${{ secrets.XRAY_CLIENT_ID }}
password: ${{ secrets.XRAY_SECRET }}
testFormat: "junit"
testPaths: "**/results/*.xml"
testExecutionJson: "cypress/test-exec/QA-test-execution.json"
testEnvironments: "QA, DEV"
projectKey: "PRJIND"
failOnImportError: true
As you can see, it tells me that it successfully exports the test results into xray, but the results are nowhere to be found.
At the end of my test runs, I do get this error
✖ 75 of 76 failed (99%) 49:49 224 1 91 - 132
Error: Process completed with exit code 1.
-the super high failure rate is expected because of 2fa. Working past that now.
I did notice on the Connect to Xray and Import Test Results sections, that each line starts with a !
Is that something to be worried about?
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
I have tried this approach before as well
- name: Submit results to Xray
uses: mikepenz/xray-action@v2
with:
xrayCloud: ${{ secrets.JIRA_SERVER_URL }}
username: ${{ secrets.XRAY_CLIENT_ID }}
password: ${{ secrets.XRAY_SECRET }}
testFormat: "junit"
testPaths: "**/results/*.xml"
testExecutionJson: "cypress/test-exec/QA-test-execution.json"
testEnvironments: "QA, DEV"
projectKey: "PRJIND"
failOnImportError: true
but with this approach I constantly get this error
Warning: 🔥 Server responded with error (Unexpected token < in JSON at position 11):
When I eventually tried the first block pasted above, it gives me the success message, but still no results in xray
Good day.
For the first log it shows undefined: https://github.com/mikepenz/xray-action/blob/main/src/processor.ts#L110 for the result.
What do you see when you enable debug logging?
Imports work if you use the Xray APi directly?
Mike
I did manage to turn on debug logging, and here is what I am seeing
ℹ️ Merged 86 junit test reports into a single file: C:\Users\RUNNER1\AppData\Local\Temp\tmp.1u9khspktk.xml1\AppData\Local\Temp\tmp.1u9khspktk.xml
##[debug]Try to import: C:\Users\RUNNER
##[debug]Using endpoint: https://xray.cloud.getxray.app//api/v2/import/execution/junit
Warning: 🔥 Failed to import: C:\Users\RUNNER~1\AppData\Local\Temp\tmp.1u9khspktk.xml (Timeout awaiting 'request' for 60000ms)
ℹ️ Processed 0 of 1 elements. Failed to import: 1
Error: 🔥 1 failed imports detected
here is the code that is generating that response
- name: Submit results to Xray
uses: mikepenz/xray-action@v2.4.5
with:
username: ${{ secrets.XRAY_CLIENT_ID }}
password: ${{ secrets.XRAY_SECRET }}
testFormat: "junit"
testPaths: "**/test-results/*.xml"
# testExecutionJson: "cypress/test-exec/QA-test-execution.json"
testEnvironments: "QA, DEV"
projectKey: "PRJIND"
failOnImportError: true
Thank you for providing additional details.
Have you verified that the upload of the given test file works using the API directly?
Additionally how large (how many test cases are there in this file)? Looks your last run, ran into a timeout after a whole minute.
Mike
I actually got it to work last night. After enabling debug logging (thanks for that tip, btw) I was able to chase down each error and solve them till I got it to work.
The issues I ran across where:
1.) it threw an error when I used the testEnvironments field. Said it did not recognize that field.
2.) It asked for a correct xray issue type ID. When I finally added that in my json, it successfully imported the tests.
Right now the test results -before compiling down to a single file- are 62 KB. We have about 100 tests so far, but expect many more. Should we expect any errors with hundreds of tests?
If I may add, the documentation is not particularly clear when it comes to the test json file. If it explained what the ID for the jira issue type is, that it is required, and how to get it, that would be immensely helpful.
Thanks for your response.
That's great to hear!
More than happy to take PRs to improve the documentation and clarify sections which are not clear.
Most of the specifics are related to XRAY though and can be found here : https://github.com/mikepenz/xray-action#xray (as the action itself mostly does the HTTP requests towards their API)