Connection gets closed but connection_lost is not called
r00ta opened this issue · 1 comments
- uvloop version: 0.14
- Python version: 3.8
- Platform: Ubuntu 20.04
- Can you reproduce the bug with
PYTHONASYNCIODEBUG
in env?: no - Does uvloop behave differently from vanilla asyncio? How?: Yes
We are using twisted with uvloop as event loop implementation.
On application side, we use the connection_lost
and connection_made
callback from asyncio.BaseProtocol
to keep track of the connections with the clients.
Under heavy load, we've hit
File "/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 173, in addWriter
self._asyncioEventloop.add_writer(fd, callWithLogger, writer,
File "uvloop/loop.pyx", line 2399, in uvloop.loop.Loop.add_writer
File "uvloop/loop.pyx", line 808, in uvloop.loop.Loop._add_writer
File "uvloop/handles/poll.pyx", line 122, in uvloop.loop.UVPoll.start_writing
File "uvloop/handles/poll.pyx", line 39, in uvloop.loop.UVPoll._poll_start
File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive
builtins.RuntimeError: unable to perform operation on <UVPoll closed=True 0x7fe32f0f9cf0>; the handler is closed
and given that connection_lost
was not called, we ended up in an infinite loop trying to reuse this closed connection.
I've investigated a bit the uvloop code and from my understanding libuv
might raise silent exceptions that can close the poll without calling the connection_lost
callback.
For example here
uvloop/uvloop/handles/poll.pyx
Line 44 in 7783f1c
cdef inline _poll_start(self, int flags):
cdef int err
print('checking alive')
self._ensure_alive()
err = uv.uv_poll_start(
<uv.uv_poll_t*>self._handle,
flags,
__on_uvpoll_event)
and here
uvloop/uvloop/handles/poll.pyx
Line 204 in 7783f1c
cdef void __on_uvpoll_event(uv.uv_poll_t* handle,
int status, int events) with gil:
if __ensure_handle_data(<uv.uv_handle_t*>handle, "UVPoll callback") == 0:
return
cdef:
UVPoll poll = <UVPoll> handle.data
if status < 0:
exc = convert_error(status)
poll._fatal_error(exc, False)
return
as poll._fatal_error
calls _close()
directly.
I've managed to (kind of) reproduce this by patching uvloop, calling directly _fatal_error
somewhere in the code and assessing that the connection_lost
is not called.
Could you double check if my understanding is correct? I've being trying to force libuv to raise such exception, but I did not find a way to do it.
Looking at asyncio docs, connection_lost
should be called also when the connection is closed https://docs.python.org/3/library/asyncio-protocol.html#asyncio.BaseProtocol.connection_lost
Thank you!
Closing this as I've found that the issue was somewhere else.