Deprecate asyncio policy system

Question

Deprecate asyncio policy system

Closed this issue 10 months ago · 15 comments

asyncio's policy system deprecation

asyncio's policy system¹ has been a source of confusion and problems in asyncio for a very long time. The policies no longer serve a real purpose. Loops are always per thread, there is no need to have a "current loop" when no loop is currently running. This issue discusses the changes to deprecate it in Python 3.14 and schedule its removal in 3.16 or later.

The usual user applications would use the runner APIs (see flowchart) while those who want more control like Jupyter project can create an event loop and manage it themselves, the difference would be that instead of them first getting the policy then event loop then can directly create it like

loop = MyCustomEventLoop()
loop.run_until_complete(task)

rather than currently

asyncio.set_event_loop_policy(MyPolicy())
policy = asyncio.get_event_loop_policy()
loop = policy.new_event_loop()
loop.run_until_complete(task)

See these discussions for more background:

Functions and classes to be deprecated and later removed

asyncio.get_event_loop_policy
asyncio.set_event_loop_policy
asyncio.AbstractEventLoopPolicy
asyncio.DefaultEventLoopPolicy
asyncio.WindowsSelectorEventLoopPolicy
asyncio.WindowsProactorEventLoopPolicy
asyncio.set_event_loop

Functions to be modified

asyncio.get_event_loop - In 3.16 or later this will become an alias to get_running_loop.
asyncio.new_event_loop - In 3.16 or later this will ignore custom policies and will be an alias to asyncio.EventLoop
asyncio.run & asyncio.Runner - In 3.16 or later this will be modified to not use policy system as that will be gone and rely solely on loop_factory.

The Grand Plan

To minimize changes, all the deprecated functions will be underscored i.e. set_event_loop -> _set_event_loop and set_event_loop will emit the warning then call _set_event_loop as its underlying implementation. This way internally asyncio can still call these functions until they are removed without need of many ignore warnings and the tests too can easily be adapted.
The deprecated classes will emit warnings when they are subclassed as it was done for child watchers.
The runner APIs will be remain unmodified but making sure that correct warnings are emitted internally when policy system is used.

The Future

---
title: Flowchart for asyncio.run
---
flowchart TD
    A["asyncio.run(coro, loop_factory=...)"] --> B{loop_factory}
    B -->|loop_factory is None| D{platform}
    B -->|loop_factory is not None| E["loop = loop_factory()"]
    D --> |Unix| F["loop = SelectorEventLoop()"]
    D --> |Windows| G["loop = ProactorEventLoop()"]
    E --> H
    F --> H
    G --> H 
    H["loop.run_until_complete(coro)"]

Linked PRs

https://docs.python.org/3.14/library/asyncio-policy.html ↩

Answer 1 · 2024-12-14T21:20:15.000Z

asyncio.new_event_loop could just be deprecated in favour of asyncio.EventLoop

Answer 2 · 2024-12-14T22:05:14.000Z

I support the deprecation in general, details are the subject for future discussion.
For example, policies are used by very many libraries. It's better for, e.g., maintainers of pytest-asyncio if we keep the deprecation for 5 Python releases (the issue proposes 2 years).
Let me sleep on other proposed details.
But anyway, I agree we should deprecate policies at least in a very soft form. If we agree on the 5 year deprecation period we can increase the pressure release-to-release

Answer 3 · 2024-12-15T07:20:31.000Z

I support the deprecation in general, details are the subject for future discussion. For example, policies are used by very many libraries. It's better for, e.g., maintainers of pytest-asyncio if we keep the deprecation for 5 Python releases (the issue proposes 2 years). Let me sleep on other proposed details. But anyway, I agree we should deprecate policies at least in a very soft form. If we agree on the 5 year deprecation period we can increase the pressure release-to-release

how about in 2 years (2 Python releases so 3.16 or 3.27) we swap the default loop_factory for asyncio.Runner and asyncio.run to loop_factory=asyncio.EventLoop This would allow policy users like pytest-asyncio to continue to use asyncio.get_event_loop() with the policy system, but would move most users off the system

Answer 4 · 2024-12-15T15:42:16.000Z

FTR, asyncio.get_event_loop ¹ in Python versions 3.10.0–3.10.8 and 3.11.0, an incorrect deprecation was added for this but it was backed out later because things like asyncio.set_event_loop_policy weren't deprecated at that time.

With the current plan all of this would be deprecated at the same time so there won't be any issue like that.

https://docs.python.org/3.11/library/asyncio-eventloop.html#asyncio.get_event_loop check the note ↩

Answer 5 · 2024-12-21T19:43:22.000Z

I'm now getting dozens of get_event_loop_policy and get_event_loop_policy deprecation warnings when running ./python.exe -m test on macOS. Please could you silence them in tests?

Details

...
0:00:07 load avg: 2.83 [  5/481] test.test_asyncio.test_events
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2875: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2878: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2893: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2896: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2916: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2920: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2930: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  self.assertIs(asyncio.get_event_loop(), loop)
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_events.py:2934: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  asyncio.get_event_loop()
0:00:13 load avg: 2.68 [  6/481] test.test_asyncio.test_futures
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_futures.py:126: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  return self.cls(*args, **kwargs)
/Users/hugo/github/python/cpython/main/Lib/asyncio/futures.py:412: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_futures.py:126: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  return self.cls(*args, **kwargs)
/Users/hugo/github/python/cpython/main/Lib/asyncio/futures.py:412: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/futures.py:79: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  self._loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/futures.py:412: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
0:00:14 load avg: 2.68 [  7/481] test.test_asyncio.test_futures2
0:00:14 load avg: 2.68 [  8/481] test.test_asyncio.test_locks
0:00:15 load avg: 2.63 [  9/481] test.test_asyncio.test_pep492
0:00:15 load avg: 2.63 [ 10/481] test.test_asyncio.test_proactor_events
0:00:16 load avg: 2.63 [ 11/481] test.test_asyncio.test_protocols
0:00:16 load avg: 2.63 [ 12/481] test.test_asyncio.test_queues
0:00:16 load avg: 2.63 [ 13/481] test.test_asyncio.test_runners
0:00:16 load avg: 2.63 [ 14/481] test.test_asyncio.test_selector_events
0:00:17 load avg: 2.63 [ 15/481] test.test_asyncio.test_sendfile
0:00:18 load avg: 2.63 [ 16/481] test.test_asyncio.test_server
0:00:18 load avg: 2.63 [ 17/481] test.test_asyncio.test_sock_lowlevel
0:00:20 load avg: 2.74 [ 18/481] test.test_asyncio.test_ssl
0:00:27 load avg: 2.52 [ 19/481] test.test_asyncio.test_sslproto
0:00:32 load avg: 2.32 [ 20/481] test.test_asyncio.test_staggered
0:00:32 load avg: 2.32 [ 21/481] test.test_asyncio.test_streams
/Users/hugo/github/python/cpython/main/Lib/asyncio/streams.py:425: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  self._loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/streams.py:128: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  self._loop = events.get_event_loop()
0:00:33 load avg: 2.32 [ 22/481] test.test_asyncio.test_subprocess
0:00:36 load avg: 2.21 [ 23/481] test.test_asyncio.test_taskgroups
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_taskgroups.py:842: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = asyncio.get_event_loop()
0:00:42 load avg: 2.19 [ 24/481] test.test_asyncio.test_tasks
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:800: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:566: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/tasks.py:731: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = events.get_event_loop()
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
0:00:58 load avg: 2.07 [ 25/481] test.test_asyncio.test_threads
0:00:58 load avg: 2.07 [ 26/481] test.test_asyncio.test_timeouts
0:00:58 load avg: 2.07 [ 27/481] test.test_asyncio.test_transports
0:00:58 load avg: 2.07 [ 28/481] test.test_asyncio.test_unix_events
/Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_unix_events.py:1198: DeprecationWarning: 'asyncio.get_event_loop_policy' is deprecated and slated for removal in Python 3.16
  loop = asyncio.get_event_loop()
...

And is the What's New entry to follow? Tip: add it to Doc/deprecations/pending-removal-in-3.16.rst, it'll be included in both the What's New and deprecations page.

Answer 6 · 2024-12-21T19:48:49.000Z

IIRC, Kumar said that he will defer docs and what's new when all the work is completed so to avoid many NEWS entries. For the tests, I think we forgot to suppress those that were not explicitly asyncio.{get,set}_* =/

Answer 7 · 2024-12-21T20:13:22.000Z

IIRC, Kumar said that he will defer docs and what's new when all the work is completed so to avoid many NEWS entries.

👍

For the tests, I think we forgot to suppress those that were not explicitly asyncio.{get,set}_* =/

Actually, I just made sure I was on latest main then cleaned and rebuilt and those deprecation warnings have gone away, sorry for the false alarm. But there are still some resource warnings:

Details

...
0:00:42 load avg: 2.46 [ 24/481] test.test_asyncio.test_tasks
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
0:00:58 load avg: 3.44 [ 25/481] test.test_asyncio.test_threads
...

Answer 8 · 2024-12-22T10:11:51.000Z

IIRC, Kumar said that he will defer docs and what's new when all the work is completed so to avoid many NEWS entries.

👍

For the tests, I think we forgot to suppress those that were not explicitly asyncio.{get,set}_* =/

Actually, I just made sure I was on latest main then cleaned and rebuilt and those deprecation warnings have gone away, sorry for the false alarm. But there are still some resource warnings:
Details

...
0:00:42 load avg: 2.46 [ 24/481] test.test_asyncio.test_tasks
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name=None coro=<coroutine_function() done, defined at /Users/hugo/github/python/cpython/main/Lib/test/test_asyncio/test_tasks.py:30>>
/Users/hugo/github/python/cpython/main/Lib/asyncio/base_events.py:759: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  _warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
0:00:58 load avg: 3.44 [ 25/481] test.test_asyncio.test_threads
...

I tracked these down using tracemalloc #128172

Answer 9 · 2024-12-22T10:16:34.000Z

@hugovk it would be good to configure warnings as errors in the tests, and collect unraisable exceptions like how pytest does it

Answer 10 · 2024-12-22T14:18:57.000Z

@hugovk it would be good to configure warnings as errors in the tests, and collect unraisable exceptions like how pytest does it

Would you like to open a new issue about it? Perhaps under https://github.com/python/core-workflow?

Answer 11 · 2024-12-24T06:55:33.000Z

I used to run ./python -We -m test for catching warning messages.

Hmm, test_inspect.py should be modified to avoid deprecated api:

0:04:22 load avg: 2.66 [ 51/481] test.test_inspect.test_inspect
test test.test_inspect.test_inspect failed -- Traceback (most recent call last):
  File "/home/andrew/projects/cpython/Lib/asyncio/events.py", line 785, in set_event_loop_policy
    warnings._deprecated('asyncio.set_event_loop_policy', remove=(3,16))
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/andrew/projects/cpython/Lib/warnings.py", line 668, in _deprecated
    warn(msg, DeprecationWarning, stacklevel=3)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
DeprecationWarning: 'asyncio.set_event_loop_policy' is deprecated and slated for removal in Python 3.16

Answer 12 · 2024-12-25T09:05:03.000Z

@hugovk it would be good to configure warnings as errors in the tests, and collect unraisable exceptions like how pytest does it

Would you like to open a new issue about it? Perhaps under python/core-workflow?

I've opened a discussion here: https://discuss.python.org/t/its-too-easy-to-introduce-noise-in-cpythons-test-suite-output-resourcewarnings-and-deprecationwarnings/75234 because that's what the core-workflow github issue template suggested

Answer 13 · 2024-12-26T14:35:15.000Z

Completed as all the deprecations and docs are added.

Answer 14 · 2024-12-26T18:53:10.000Z