Flaky tests, because exceptions in tests can be ignored
Opened this issue · 0 comments
UncleInf commented
When running tests with cucumber/serenity runner exceptions can be ignored, resulting in runs which pass but that should have failed. To reproduce there is a repo fork with branch with a test to reproduce the issue.
Exceptions in tests do get ignored when:
@Step
annotation is used in single scenario- or atleast one scenario uses
@Step
annotations in feature file run
Attaching a picture from 2 runs:
- both scenarios DOES throw exception
- when ran as a single test without
@Step
annotation -- test fails (picture on the left) - when ran as a whole feature -- tests pass even though both tests should fail (picture on the right)