0.25.1 regression Python 3.9 asyncio_mode="auto" # RuntimeError: There is no current event loop in thread 'MainThread'
dimaqq opened this issue · 6 comments
Happy new year.
0.25.1 was released recently and somehow it fails on Python 3.9 specifically.
In my case, asyncio_mode = "auto" is used, so maybe similar to #658, however I don't know if that's required for the regression.
Meanwhile, Python 3.8 is OK (0.25.x can't be used, 0.24 is used instead)
And Python 3.10, 3.11, 3.12, 3.13 are OK (with the new version)
Version resolution:
{ name = "pytest-asyncio", version = "0.24.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.9'" },
{ name = "pytest-asyncio", version = "0.25.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.9'" },Run log: https://github.com/juju/python-libjuju/actions/runs/12423507633
Screenshot for the above:
I've manually validated that 0.25.0 works fine in my setup.
Log, if helps any:
self = <tests.unit.test_model.TestModelState testMethod=test_apply_delta>
def test_apply_delta(self):
> model = Model()
tests/unit/test_model.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
juju/model/__init__.py:677: in __init__
self._watch_stopping = asyncio.Event()
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/locks.py:177: in __init__
self._loop = events.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x105a521f0>
def get_event_loop(self):
"""Get the event loop for the current context.
Returns an instance of EventLoop or raises an exception.
"""
if (self._local._loop is None and
not self._local._set_called and
threading.current_thread() is threading.main_thread()):
self.set_event_loop(self.new_event_loop())
if self._local._loop is None:
> raise RuntimeError('There is no current event loop in thread %r.'
% threading.current_thread().name)
E RuntimeError: There is no current event loop in thread 'MainThread'.My config, since the most likely cause is the change that fixed interleaved loop scopes:
[tool.pytest.ini_options]
addopts = "-ra -v"
asyncio_default_fixture_loop_scope = "function"
asyncio_mode = "auto"
filterwarnings = "ignore::DeprecationWarning:websockets"
markers = [
"serial: mark a test that must run by itself",
"wait_for_idle: mark a test that waits for the model to be idle",
"bundle: mark a test that uses a bundle",
]
I don't have custom loop_scope's in the tests, only this config.
I do, however, have a mix of modern pytest tests (module-level test_foo(): ...) and older class-based tests class TestFoo(unittest.TestCase): def test_foo(self): ...)
Easy reproducer:
git clone git@github.com:juju/python-libjuju.git
cd python-libjuju
# currently at a58645e
uvx -p 3.9 tox -e unitThanks for the detailed report!
The failing test is a synchronous test. Maybe #1029 introduced a regression here.
Yes the test is synchronous, however it instantiates some code which in turn creates an asyncio.Event().
Also have received a similar traceback for this test definition:
@pytest.mark.asyncio()
async def test_global_retry_config_disable_async():
s = SDK(retry_config=None)
with pytest.raises(
errors.APIError, match="API error occurred: Status 503"
) as exc_info:
await s.retries.retries_get_async(request_id=str(uuid.uuid4()), num_retries=2)
assert exc_info.value.status_code == 503 [gw0] linux -- Python 3.9.21 /home/runner/work/xxx/.venv/bin/python
self = <Coroutine test_global_retry_config_disable_async>
def runtest(self) -> None:
self.obj = wrap_in_sync(
# https://github.com/pytest-dev/pytest-asyncio/issues/596
self.obj, # type: ignore[has-type]
)
> super().runtest()
.venv/lib/python3.9/site-packages/pytest_asyncio/plugin.py:533:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.9/site-packages/pytest_asyncio/plugin.py:1049: in inner
_loop = _get_event_loop_no_warn()
.venv/lib/python3.9/site-packages/pytest_asyncio/plugin.py:979: in _get_event_loop_no_warn
return asyncio.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fd55aef8ee0>
def get_event_loop(self):
"""Get the event loop for the current context.
Returns an instance of EventLoop or raises an exception.
"""
if (self._local._loop is None and
not self._local._set_called and
threading.current_thread() is threading.main_thread()):
self.set_event_loop(self.new_event_loop())
if self._local._loop is None:
> raise RuntimeError('There is no current event loop in thread %r.'
% threading.current_thread().name)
E RuntimeError: There is no current event loop in thread 'MainThread'.
/opt/hostedtoolcache/Python/3.9.21/x64/lib/python3.9/asyncio/events.py:642: RuntimeError
I found some time to dig into this. Let me outline my findings.
As @dimaqq already mentioned, the failing test calls asyncio.Event() and only fails on Python 3.9. The implementation of the Event constructor in CPython 3.9 calls asyncio.get_event_loop() which is also seen in the stacktrace. In CPython 3.9 this function behaves differently depending on whether asyncio.set_event_loop() has been called or not: If set_event_loop has never been called before, get_event_loop will return a fresh loop. However, if set_event_loop has already been called on the current event loop policy, get_event_loop will raise the reported RuntimeError (see the docs). Later implementations of asyncio.Event() from CPython 3.10 onwards no longer call asyncio.get_event_loop(), which is why they don't produce the same error.
Indeed, running the failing test "test_apply_delta" on its own causes the test to run successfully, whereas running the full test suite makes the test fail when pytest-asyncio v0.25.1 is installed.
Therefore, I incrementally disabled tests from the test suite and found that pytest -k "test_controller or test_apply_delta" causes test_apply_delta to fail. When I inspected the test fixtures with pytest's --setup-show option, I saw that none of the tests use pytest-asyncio fixtures. The tests in the test_controller module use unittest.IsolatedAsyncioTestCase, which happens to call asyncio.set_event_loop() during the test run. This explains why the subsequent call to asyncio.Event() fails during test_apply_delta.
Most importantly, the tests produce the same error even when pytest-asyncio is disabled with -p no:asyncio:
$ tox -e unit
unit: commands[0]> pytest -x --setup-show -k 'test_controller or test_apply_delta' -p no:asyncio /tmp/python-libjuju/tests/unit
===== test session starts =====
platform linux -- Python 3.9.21, pytest-8.3.4, pluggy-1.5.0 -- /tmp/python-libjuju/.tox/py3/bin/python
cachedir: .tox/py3/.pytest_cache
rootdir: /tmp/python-libjuju
configfile: pyproject.toml
collected 260 items / 250 deselected / 10 selected
tests/unit/test_controller.py::TestControllerConnect::test_file_cred_v2
SETUP C _unittest_setUpClass_fixture_TestControllerConnect
tests/unit/test_controller.py::TestControllerConnect::test_file_cred_v2 (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_file_cred_v3
tests/unit/test_controller.py::TestControllerConnect::test_file_cred_v3 (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_no_args
tests/unit/test_controller.py::TestControllerConnect::test_no_args (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_with_controller_name
tests/unit/test_controller.py::TestControllerConnect::test_with_controller_name (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_bakery_client
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_bakery_client (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_macaroons
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_macaroons (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_no_auth
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_no_auth (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_userpass
tests/unit/test_controller.py::TestControllerConnect::test_with_endpoint_and_userpass (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
tests/unit/test_controller.py::TestControllerConnect::test_with_posargs
tests/unit/test_controller.py::TestControllerConnect::test_with_posargs (fixtures used: _unittest_setUpClass_fixture_TestControllerConnect, request)PASSED
TEARDOWN C _unittest_setUpClass_fixture_TestControllerConnect
tests/unit/test_model.py::TestModelState::test_apply_delta
SETUP C _unittest_setUpClass_fixture_TestModelState
tests/unit/test_model.py::TestModelState::test_apply_delta (fixtures used: _unittest_setUpClass_fixture_TestModelState, request)FAILED
TEARDOWN C _unittest_setUpClass_fixture_TestModelState
===== FAILURES =====
_____ TestModelState.test_apply_delta _____
self = <tests.unit.test_model.TestModelState testMethod=test_apply_delta>
def test_apply_delta(self):
> model = Model()
tests/unit/test_model.py:72:
_ _ _
juju/model/__init__.py:677: in __init__
self._watch_stopping = asyncio.Event()
/usr/lib/python3.9/asyncio/locks.py:177: in __init__
self._loop = events.get_event_loop()
_ _ _
self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7f00fa9d7fa0>
def get_event_loop(self):
"""Get the event loop for the current context.
Returns an instance of EventLoop or raises an exception.
"""
if (self._local._loop is None and
not self._local._set_called and
threading.current_thread() is threading.main_thread()):
self.set_event_loop(self.new_event_loop())
if self._local._loop is None:
> raise RuntimeError('There is no current event loop in thread %r.'
% threading.current_thread().name)
E RuntimeError: There is no current event loop in thread 'MainThread'.
/usr/lib/python3.9/asyncio/events.py:642: RuntimeError
===== warnings summary =====
.tox/py3/lib/python3.9/site-packages/_pytest/config/__init__.py:1441
/tmp/python-libjuju/.tox/py3/lib/python3.9/site-packages/_pytest/config/__init__.py:1441: PytestConfigWarning: Unknown config option: asyncio_default_fixture_loop_scope
self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")
.tox/py3/lib/python3.9/site-packages/_pytest/config/__init__.py:1441
/tmp/python-libjuju/.tox/py3/lib/python3.9/site-packages/_pytest/config/__init__.py:1441: PytestConfigWarning: Unknown config option: asyncio_mode
self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")
juju/client/connection.py:31
/tmp/python-libjuju/juju/client/connection.py:31: DeprecationWarning: websockets.WebSocketClientProtocol is deprecated
_WebSocket: TypeAlias = websockets.WebSocketClientProtocol
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===== short test summary info =====
FAILED tests/unit/test_model.py::TestModelState::test_apply_delta - RuntimeError: There is no current event loop in thread 'MainThread'.
!!!!! stopping after 1 failures !!!!!
===== 1 failed, 9 passed, 250 deselected, 3 warnings in 0.66s =====
unit: exit 1 (0.90 seconds) /tmp/python-libjuju> pytest -x --setup-show -k 'test_controller or test_apply_delta' -p no:asyncio /tmp/python-libjuju/tests/unit pid=25775
unit: FAIL code 1 (0.94=setup[0.04]+cmd[0.90] seconds)
evaluation failed :( (1.02 seconds)
(
Therefore, I conclude that this is not a pytest-asyncio bug. The issue in the test suite was simply overshadowed by bad behavior of pytest-asyncio in earlier versions.
Changing line 70 of tests/unit/test_model.py from class TestModelState(unittest.TestCase): to class TestModelState(unittest.IsolatedAsyncioTestCase): fixes the issue on Python 3.9.
@bflad Your issue is likely a different one. The best way to solve it is to provide a minimal reproducible example in a separate issue.