Serenity Run using Cucumber Tags - Serenity Run Console indicates tests for ALL TAGS executed even when Single TAG is executed (Serenity Reports as expected)
TarahMo opened this issue · 7 comments
Hi John @wakaleo
I am noticing a strange behavior during build run-time within Serenity Console Run when running Serenity Tests using Cucumber Tags.
I have 6 Scenarios in the Feature File each with different Tags. When I run the project using Maven run configuration for a specific TAG e.g. for "GETSERVICES" TAG (for the highlighted scenario in the attached files), observations highlighted under 1) & 2) below.
-
When the build execution starts - I find it ambiguous looking at the Serenity Run Console as it indicates 6 TESTS Executed & 6 TESTS Passed. I have used log().all() in the REST Assured code which is logging the request & response and I can see ALL 6 API's have been executed.
-
Build is successful (Process finished with exit code 0 & BUILD SUCCESS) and when I look at "Serenity Summary Emailable Report" & "Index.HTML Report" -> as expected, I see ONLY 1 TEST (based on the TAG provided) executed & PASSED.
Therefore I am confused why Serenity Run Console indicates 6 TESTS Executed & 6 TESTS Passed & it actually does and I see GET & POST responses for ALL 6 API's in the console logs.
I have attached a few screen prints for the following files & have attached the Full Console Log as well.
Please can you advise what may be the reason for the above behavior?
Attached screen prints & logs for the following:
i) Serenity Run - Console Logs
ii) Serenity Run Logs
iii) Feature File
iv) Serenity Runner File
v) Maven Run Configuration
vi) Index.html Report
vii) serenity-summary.html (Serenity Summary Emailable Report)
viii) POM.XML Maven Failsafe Plugin
ix) POM.XML (full)
Serenity Run - Console Logs.txt
Looks like a bug.
Hi John @wakaleo
do you mean a bug in serenity using cucumber with tags? I have spent quite a lot of time looking at the pom & my code & the serenity documentation though everything looks alright to me. I have also added 'serenity.exclude.unrelated.requirements.of.type = capabilities' under serenity.conf but that didn't resolve the issue.
Do you mind taking a quick look at the screenshots I attached above if you see anything that's not right?
Probably a bug in the Serenity code.
Can we probably clock/register it somewhere?
Yes, but raise this issue on https://github.com/serenity-bdd/serenity-core rather than here.
Sure John. I have created the above issue under serenity-core and shared the project details. Closing this issue here.