pytest unittest cases are destroyed after first teardown
Chris-Peterson444 opened this issue · 0 comments
Chris-Peterson444 commented
The following test from test/test_pytest/test_pytest_example.py
fails:
@flaky
class TestExampleFlakyTestCase(TestCase):
_threshold = -1
@staticmethod
def test_flaky_thing_that_fails_then_succeeds():
"""
Flaky will run this test twice.
It will fail once and then succeed.
"""
TestExampleFlakyTestCase._threshold += 1
assert TestExampleFlakyTestCase._threshold >= 1
========================================================================================================================== FAILURES ==========================================================================================================================
_____________________________________________________________________________________________ TestExampleFlakyTestCase.test_flaky_thing_that_fails_then_succeeds _____________________________________________________________________________________________
> lambda: ihook(item=item, **kwds), when=when, reraise=reraise
)
E AssertionError
flaky/flaky_pytest_plugin.py:148: AssertionError
===Flaky Test Report===
test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExampleFlakyTestCase._threshold
[<TracebackEntry /usr/lib/python3.12/unittest/case.py:58>, <TracebackEntry /usr/lib/python3.12/unittest/case.py:634>, <TracebackEntry /usr/lib/python3.12/unittest/case.py:589>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/test/test_pytest/test_pytest_example.py:101>]
test_flaky_thing_that_fails_then_succeeds failed; it passed 0 out of the required 1 times.
<class 'AssertionError'>
[<TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/runner.py:342>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/flaky/flaky_pytest_plugin.py:148>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_hooks.py:513>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_manager.py:120>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:139>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:122>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:122>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:122>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/logging.py:850>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/logging.py:833>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:122>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/capture.py:878>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:122>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/skipping.py:257>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/pluggy/_callers.py:103>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/runner.py:184>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/runner.py:174>, <TracebackEntry /<working-dir>/python-flaky-3.8.1/venv/lib/python3.12/site-packages/_pytest/unittest.py:319>]
===End Flaky Test Report===
================================================================================================================== short test summary info ===================================================================================================================
FAILED test/test_pytest/test_pytest_example.py::TestExampleFlakyTestCase::test_flaky_thing_that_fails_then_succeeds - AssertionError
===================================================================================================================== 1 failed in 0.04s ======================================================================================================================
The first iteration fails (expected) and then each subsequent iteration fails due to an assertion error in pytest:
def runtest(self) -> None:
from _pytest.debugging import maybe_wrap_pytest_function_for_tracing
testcase = self.instance
> assert testcase is not None
E AssertionError
This appears to be due to a bug in pytest itself pytest-dev/pytest#12424