pytest-dev/pytest-github-actions-annotate-failures

0.1.4 breaks CI: AttributeError: 'Function' object has no attribute 'execution_count'

frenck opened this issue · 5 comments

frenck commented

0.1.4 breaks our test suite on GitHub actions:

Arguments: ('Future exception was never retrieved',)
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/xdist/remote.py", line 87, in pytest_runtestloop
INTERNALERROR> E                 self.run_one_test(torun)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/xdist/remote.py", line 106, in run_one_test
INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR> E                 return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR> E                 self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
INTERNALERROR> E                 runtestprotocol(item, nextitem=nextitem)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/_pytest/runner.py", line 120, in runtestprotocol
INTERNALERROR> E                 rep = call_and_report(item, "setup", log)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/_pytest/runner.py", line 217, in call_and_report
INTERNALERROR> E                 report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR> E                 return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR> E                 self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR> E                 gen.send(outcome)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pytest_github_actions_annotate_failures/plugin.py", line 34, in pytest_runtest_makereport
INTERNALERROR> E                 if item.execution_count <= pytest_rerunfailures.get_reruns_count(item):
INTERNALERROR> E             AttributeError: 'Function' object has no attribute 'execution_count'
INTERNALERROR> E           assert False

Ref: home-assistant/core#58351

marvin-w commented

#40 (well the release of it) caused the pipelines to fail.

It looks like the import check does not work correctly. Home Assistant does not use the pytest-rerunfailures plugin (at least it's not defined in any requirement and is also not manually installed during GH Action execution.

Maybe checking if execution_count exists on item is a better approach to handle this (and also saves the dirty pass of the ImportError).

execution_count is set here: https://github.com/pytest-dev/pytest-rerunfailures/blob/master/pytest_rerunfailures.py#L487

Also, one could argue that its better to fix the flaky tests instead of rerunning them in CI - but I guess that's a philosophy question.

  • 👍3
henryiii commented

You should be able to check for the entry point to detect the plugin, that's how PyTest discovers plugins. Worth checking, anyway.

utgwkk commented

I've just released version 0.1.5 (reverted changes of 0.1.4).

billyvg commented

Ah sorry about this - I had manually tested this and it seemed to work, will look into a different method of detecting the plugin.

utgwkk commented

Closing this issue because CI error was resolved by reverting changes.

  • 👍2