Allure report treats passed Mocha beforeAll and afterAll hooks as passed tests
MartijnThiele opened this issue · 20 comments
I am building a ta-framework using
-- wdio-mocha-framework@0.6.2 -- wdio-allure-reporter@0.6.3
`-- webdriverio@4.13.1
My framework has around 20 spec files, most of which make use of mocha's beforeAll and afterAll hooks to do setup and cleanup.
My Allure report (generated by jenkins) sees these hooks as actual tests, reporting them as such. This causes allure to report almost 40 extra tests, which messes up my report
I also found a variation to this behaviour: if a spec file contains one or more skipped tests, the beforeAll method will show up as skipped also. This could have something to do with the freshly fixed bug that causes only 1 skipped test to be shown, but I am not sure.
For reference, the images below are allure report and wdio spec report for the same test run.
Could the wdio allure reporter be adjusted so that beforeAll and AfterAll hooks are not shown anymore when passing? (a failing hook is a different story, the default mocha way of handling these could use some improvement too)
create sample project on github to help me reproduce these issues
created a sample project here:
https://github.com/MartijnThiele/wdioAllureSampleProjectIssue-128
Thanks for taking the time to look into this.
Please let me know if there are any questions
@MartijnThiele thank you. I will have look into it
yes, it is a bug.
I suspected as much. Anything you can fix?
I think we can disable\enable reporting hooks at all with configuration option.
Anything you can fix?
I am not sure. I don't have plans to support this version much.
Recently I port this to v5 and will continue adding fixes\improvements only for v5
Pity, can you point me towards this configuration option? I am not sure what you mean by this.
I mean if you want to make PR to fix this bug, I suppose the best solution is to create a new option to disable reporting hooks at all.
OK, thanks for the info. That would sound like a good solution. I'm afraid I am not experienced enough to solve something like this myself, but I will have a go and see what I can do.
Hello guys! Is this issue fixed? I'm experiencing the same issue. Hooks "before all" and other are marked as test cases and including to the total number of tests.
Can this be ported?
allure-framework/allure-js@cbbb449
Can this be ported?
Feel free to provide a PR
It is fixed in v5
unfortunately still stuck on v4 for a while longer thanks for the update @BorisOsipov
still stuck on v4 for a while longer
Why is that? It is really not much work to update.
v4 is wrapped up in our own tooling suite in a way that requires more work than anticipated to move but, that was based on beta v5 last time we looked. Going to start the move / assessment again.
thanks @christian-bromann
Can this be ported?
allure-framework/allure-js@cbbb449
@brownad the commit fixed the handling for beforeEach/afterEach hooks. That was a regression in 2.0.0 version comparing to 1.x. The before and after hooks are not addressed yet.
I created an issue to track this work: allure-framework/allure-js#48
P.S. Please note that 2.0 version is not yet stable and there may be breaking changes. I advice not to use the dependency in your projects until the stable release is shipped.
It is fixed in v5
@BorisOsipov I use v5 and still have this issue
"devDependencies": {
"@wdio/allure-reporter": "^5.12.1",
"webdriverio": "^5.12.5"
}
My report:
fix @qwez relesed in v5.13.0