Feature Request(s): Labels and Test Repository Folder support
jdborneman-terminus opened this issue · 9 comments
Labels
It'd be nice if we could put a field for JIRA labels that would get applied to the tests or test executions imported by the xray-action (we would use this for things like Automation or Chrome:99 or stuff like that). I don't see labels anywhere in the documentation
Test Repository Folder
Tests show up as Test Cases in the root Test Repository folder in Xray. Would be nice to be able to specify a folder (or path) within the Test Repository in Xray to put the cases in.
If either of these exist and I missed it, apologies.
Thanks
Jason
Thank you for the report @jdborneman-terminus .
This action is supporting the xray API and their features. For specific questions around the xray API please refer to the official documentation.
The action offers the ability to provide for example a testExecutionJson and testJson with further information useable by jira / xray
- name: Prepare Xray test import
run: |
echo '{
"fields": {
"summary": "SUMMARY",
"description": "Description",
"customfield_12345": [ { "value": "XYZ" } ],
"assignee": {
"id": "xyz"
},
"issuetype": {
"id": "1234"
},
"labels": [ "some_label" ]
}
}' > ".github/config/testExecution.json"
echo '{
"fields": {
"labels": [ "some_label" ],
"customfield_12345": [ { "value": "XYZ" } ]
}
}' > ".github/config/test.json"
- name: Import Results with Xray
uses: mikepenz/xray-action@v2
with:
username: ${{ secrets.XRAY_USERNAME }}
password: ${{ secrets.XRAY_PASSWORD }}
testPaths: '*.xml'
combineInSingleTestExec: true
projectKey: "PROJ"
testPlanKey: "PROJ-1234"
testExecutionJson: '.github/config/testExecution.json'
testJson: '.github/config/test.json'
Server/DC:
https://docs.getxray.app/display/XRAY/Import+Execution+Results+-+REST
Cloud:
https://docs.getxray.app/display/XRAYCLOUD/Import+Execution+Results+-+REST
@mikepenz I'll take a deeper look at this for sure. Looks like the testJson and testExecutionJson might do what we want, I'll just have to try it out and see how it looks in practice.
Thanks a bunch!
Sounds great.
Please also be careful when it comes cloud vs server/DC as Xray functions differently in the 2 environments. Given the different APIs exposed.
The above was example for cloud
@mikepenz looks like that's working for labels for me on a Test Execution! Thanks!
But another question that's sorta related (and I'll close this issue after this 😁 ) - is there a way to set the Test's Test Details > Test Type via this module? Currently they all import as Generic and we'd like to use the type Automated that (I beleive) our Admin just added to our system. I tried searching the code for testType and other searches but came up empty. Hopefully it's something obvious I just missed. I'm guessing it's something I can add to testJson?
@jdborneman-terminus you may have a look at example 4 here: https://docs.getxray.app/display/XRAY/Import+Execution+Results
It outlines the type param inside the testInfo: "type": "Manual",
If you look into the multiPart API, it allows to provide testInfo: https://docs.getxray.app/display/XRAYCLOUD/Import+Execution+Results+-+REST#ImportExecutionResultsREST-JUnitXMLresultsMultipart
For Cloud the action sends the testInfo here: https://github.com/mikepenz/xray-action/blob/main/src/xray-cloud.ts#L100-L110
Which is the input testJson
@mikepenz Hm, ok so per
https://docs.getxray.app/display/XRAY/Import+Execution+Results#ImportExecutionResults-%22testInfo%22object-CreatingTestissues
testType is on the same level with labels
Our Jira admin added a Test Type called Automated.
Here's what I put in my test.json creation step and report step:
# Prepare Xray Test Import
- name: Prepare Xray Test Import
run: |
echo '{
"fields": {
"labels": [ "automation", "cypress" ],
"type": "Automated" # Also tried "testType": "Automated"
}
}' > ".github/test.json"
# Report Cypress Test Results To JIRA/Xray
- name: Report Cypress Test Results to JIRA/Xray
uses: mikepenz/xray-action@v2.3.1
with:
username: <<SECRET VAR NAME REDACTED>>
password: <<SECRET VAR NAME REDACTED>>
xrayBaseUrl: "https://xray.cloud.getxray.app"
testFormat: "junit"
testPaths: "apps/myapp/cypress/results/test-results.xml"
projectKey: "CYPPOC"
testPlanKey: "CYPPOC-403"
testExecKey: ${{ needs.cypress_create_test_execution.outputs.cypress_xray_test_execution }}
testEnvironments: "ci"
testJson: ".github/test.json"
But the Tests were still imported as Generic.
Then I noticed you wrote "type" as the field, not testType like is at that link. So I tried that and they still show up as Generic. Not sure WHAT I'm missing here?
(We're using Cypress parallelization, so I create a Test Execution in a previous job and require that for the actual testing and reporting job that gets split up into parallel executions. So it, in parallel, reports results from 9 distinct test-results.xml files to the same Test Execution JIRA issue. A fun thing we're also seeing is that 2 of the 9 are failing with a 400 bad request for reasons I can't figure out yet but I'm gonna focus on that AFTER I figure out the testType thing)
And you use a server/DC instance? or cloud?
Given your above config it looks like you use cloud, but the link for the import is server/DC.
For cloud it could require to be part of the xrayFields. There is sadly no direct example for cloud which I can find. (Cloud vs DC/Server behaves very different and is not interchangeable)
Please note, the action is using the official API to send the arguments as outlined in their docs. I can also only refer to their docs in this regards.
For specific usecases, it may be easier to reach out to Xray and ask where it would be meant to be sent in their API.
We're on Cloud. We reached out to them yesterday about this as well (so if I get a good answer I'll post here for future people that may find this). 🤞
Closing due to inactivity. Feel free to reopen if there is an outstanding item for the action itself.