Why are the cases skipped due to hook failure marked as failed in testrail?
praveen-nj opened this issue · 6 comments
Let's say there are N cases in a file with a before/after or beforeEach/afterEach hook and if the first test fails at the hook level cypress automatically skips the rest of em.
These tests are categorized as skipped in cypress cloud whereas the plugin reports it as a failure to Testrail.
hi there
i guess thats related to the other issue?
so you mean if cypress stops due to an exception in a hook, then all tests should be marked as skipped or blocked in testrail?
Yes, marking such cases as blocked will give us clear insights.
sry comment in wrong pr but regarding the skipping of pending tests.
i now know why it has been added.
there was a request when new runs are created that skipped ones are not being added which makes sense.
which mode are you using? adding to existing run or creating new ones?
i could add skipped ones to existing runs and avoid adding them to new ones. would that be good for you? otherwise i would need some kind of configuration
I don't understand what you're trying to say. Could you please rephrase?
Thanks in advance.
oh sorry just saw this
so.
lets say you want to create runs on the fly in TestRail.
and you have skipped tests in Cypress. Then I had the request that those skipped tests should NOT be added to the TestRail runs
But when you have an existing TestRail run, and some of the tests in Cypress are skipped, then this SHOULD of course be sent to TestRail, because it makes sense
that makes me think these 2 options
a) skipped Cypress tests are sent to TestRail on existing runs
b) skipped Cypress tests are NOT sent to TestRail when TestRail runs are created on the fly
can you live with this? :)
released in v2.8.0 :)
createMode will not add skipped test, while tests against existing TestRail Runs will also send skipped ones