Requesting loop_scope="module" (or "session" etc.) returns incorrect event_loop if another test function requests a different loop.
TimChild opened this issue · 5 comments
First of all, thanks for the great pytest plugin!
Also, I like the idea of being able to specify event_loop scopes explicitly and separately to the fixture scopes.
While playing with this, I kept running into issues that my "session" scoped event_loop seemed to change between tests that were running in different files, but only if certain other tests were run...
I think this example reproduces the issues in a single file. I recognize that the docs explicitly advise against doing this, but I hope it makes the problem easier to reason about.
Here's the example.
import pytest_asyncio
import asyncio
import pytest
class FakeAsyncConnection:
def __init__(self, loop):
self.loop = loop
async def do_something(self):
# Check if the current loop is the same as the one with which the
# connection was created
if asyncio.get_event_loop() is not self.loop:
raise RuntimeError(
"This connection is being used with a different event loop!")
return "Success"
@pytest_asyncio.fixture(scope="module", loop_scope="module")
async def async_connection():
"""Set up a async connection object with module scope."""
event_loop = asyncio.get_event_loop()
print(f"Setting up fixture: event_loop_id {id(event_loop)}")
connection = FakeAsyncConnection(event_loop)
yield connection
@pytest.mark.asyncio(loop_scope="module")
async def test_use_module_scope_loop_1(async_connection):
"""Use module loop"""
print(f"Test using loop with id: {id(asyncio.get_event_loop())}")
result = await async_connection.do_something()
assert result == "Success"
@pytest.mark.asyncio(loop_scope="module")
async def test_use_module_scope_loop_2(async_connection):
"""Use module loop again"""
print(f"Test using loop with id: {id(asyncio.get_event_loop())}")
result = await async_connection.do_something()
assert result == "Success"
@pytest.mark.asyncio(loop_scope="function")
async def test_use_function_scope_loop_1(async_connection):
"""Use function loop"""
print(f"Test using loop with id: {id(asyncio.get_event_loop())}")
with pytest.raises(RuntimeError, match="This connection is being used with a different event loop!"):
# This should raise an error because the connection is being used with a different loop
await async_connection.do_something()
@pytest.mark.asyncio(loop_scope="module")
async def test_use_module_scope_loop_3(async_connection):
"""Unexpectedly fail to use module scope again"""
print(f"Test using loop with id: {id(asyncio.get_event_loop())}")
result = await async_connection.do_something()
assert result == "Success"I would expect all tests to pass, however, the final test test_use_module_scope_loop_3 fails only if the test_use_function_scope_loop_1 is present. If the function scope one is commented out, the final test does pass (as expected).
The fixtures aren't obviously set up incorrectly (running with --setup-show):
SETUP S event_loop_policy
SETUP M tests/test_a.py::<event_loop> (fixtures used: event_loop_policy)
SETUP M async_connection
tests/test_a.py::test_use_module_scope_loop_1 (fixtures used: async_connection, event_loop_policy, request, tests/test_a.py::<event_loop>).
tests/test_a.py::test_use_module_scope_loop_2 (fixtures used: async_connection, event_loop_policy, request, tests/test_a.py::<event_loop>).
SETUP F event_loop
tests/test_a.py::test_use_function_scope_loop_1 (fixtures used: async_connection, event_loop, event_loop_policy, request).
TEARDOWN F event_loop
tests/test_a.py::test_use_module_scope_loop_3 (fixtures used: async_connection, event_loop_policy, request, tests/test_a.py::<event_loop>)F
TEARDOWN M async_connection
TEARDOWN M tests/test_a.py::<event_loop>
TEARDOWN S event_loop_policy
But for some reason I don't understand, the module scoped event loop changes for the last test.
The printed loop ids tell the same story... The fixture and first two tests all get the the same loop_id as expected. The function scope test gets a new one as expected. Then the final module scope test also gets a new loop_id (different from both previous loop_ids) unexpectedly.
Versions:
python: 3.12.7
pytest: 8.3.3
pytest-asyncio: 0.24.0
Also, my pytest settings are:
[tool.pytest.ini_options]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope="function"
Good catch and great reproducer! Thanks for the report.
The test that breaks the event loop is actually test_use_function_scope_loop_1. It uses the deprecated event_loop fixture which entails a bunch of invisible cleanups.
A fix for this should be included in a patch release for v0.24.
Hi. I'm currently trying to update our code base on python-telegram-bot to from pytest-asyncio 0.21.2 to 0.25.0, see python-telegram-bot/python-telegram-bot#4607. Unfortunately I'm running into problems which our rather large setup of differently-scoped fixtures. I tried running all tests in the session even loop using the recommendation from here and here. Unfortunately, I still get problems in some teardown, where the event loop apparently is already closed. Because the test suite is large, I haven't yet narrowed it down to one specific problematic test.
My impression is that the event loops might still not be quite correct. While browsing through the threads on this repo, I found this one here, which does show a bug in event loop selection. I notice that it was marked for the v0.25 milestone, but is not resolved so far. Indeed, running the MWE still gives a problem. Just wantod to kindly point this out and suggest to move to the next milestone 🙂
As you can see, the event_loop fixture is still being used. According to @seifertm, this fixuter is deprecated. However, the deprecation is not noted in the docs and the changelog of 0.22.0 only states that "redefinition of the event_loop fixture" is deprecated - not the event_loop itself 🤔
[tool.pytest.ini_options]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"pytest -k test_demo --setup-show -s
================================================================================== test session starts ===================================================================================
platform win32 -- Python 3.11.9, pytest-8.3.4, pluggy-1.5.0
rootdir: C:\Users\hinri\PycharmProjects\python-telegram-bot
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.4.0, flaky-3.8.1, asyncio-0.25.0, socket-0.7.0, xdist-3.6.1, web3-6.18.0
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=function
collected 6239 items / 6235 deselected / 4 selected
tests\test_demo.py
SETUP S event_loop_policy
SETUP M tests/test_demo.py::<event_loop> (fixtures used: event_loop_policy)Setting up fixture: event_loop_id 2321343405904
SETUP M async_connection
tests/test_demo.py::test_use_module_scope_loop_1 (fixtures used: async_connection, event_loop_policy, request, tests/test_demo.py::<event_loop>)Test using loop with id: 2321343405904
.
tests/test_demo.py::test_use_module_scope_loop_2 (fixtures used: async_connection, event_loop_policy, request, tests/test_demo.py::<event_loop>)Test using loop with id: 2321343405904
.
SETUP F event_loop
tests/test_demo.py::test_use_function_scope_loop_1 (fixtures used: async_connection, event_loop, event_loop_policy, request)Test using loop with id: 2321343010000
.
TEARDOWN F event_loop
tests/test_demo.py::test_use_module_scope_loop_3 (fixtures used: async_connection, event_loop_policy, request, tests/test_demo.py::<event_loop>)Test using loop with id: 2321343284624
F
TEARDOWN M async_connection
TEARDOWN M tests/test_demo.py::<event_loop>
TEARDOWN S event_loop_policy
======================================================================================== FAILURES ========================================================================================
______________________________________________________________________________ test_use_module_scope_loop_3 ______________________________________________________________________________
async_connection = <tests.test_demo.FakeAsyncConnection object at 0x0000021C7AD0E3D0>
@pytest.mark.asyncio(loop_scope="module")
async def test_use_module_scope_loop_3(async_connection):
"""Unexpectedly fail to use module scope again"""
print(f"Test using loop with id: {id(asyncio.get_event_loop())}")
> result = await async_connection.do_something()
tests\test_demo.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <tests.test_demo.FakeAsyncConnection object at 0x0000021C7AD0E3D0>
async def do_something(self):
# Check if the current loop is the same as the one with which the
# connection was created
if asyncio.get_event_loop() is not self.loop:
> raise RuntimeError(
"This connection is being used with a different event loop!")
E RuntimeError: This connection is being used with a different event loop!
tests\test_demo.py:14: RuntimeError
====================================================================== 1 failed, 3 passed, 6235 deselected in 5.51s ======================================================================@Bibo-Joshi Your point about the docs is absolutely valid, thanks for raising it.
We should track the documentation issue as part of #375 .
Thanks for the confirmation! I guess you meant to link to a different thread, though? :D #375 was merged three years ago ...
@Bibo-Joshi Sorry, I meant #964.
Both issues have a remarkably similar title :)