RuntimeError: Cannot pause_reading() when closing, triggered via FlowControlStreamReader.feed_data
Closed this issue · 7 comments
zmedico commented
Long story short
When trying to read the response to a get request from an https endpoint, I get a RuntimeError: Cannot pause_reading() when closing exception.
Expected behaviour
I expect it to read the response successfully.
Actual behaviour
File "/usr/lib64/python3.5/site-packages/aiohttp/client_reqrep.py", line 739, in read
self._content = yield from self.content.read()
File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 507, in wrapper
result = yield from func(self, *args, **kw)
File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 562, in read
return (yield from super().read(n))
File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 265, in read
raise self._exception
File "/usr/lib64/python3.5/site-packages/aiohttp/parsers.py", line 190, in set_parser
next(p)
File "/usr/lib64/python3.5/site-packages/aiohttp/protocol.py", line 313, in __call__
yield from self.parse_length_payload(out, buf, length)
File "/usr/lib64/python3.5/site-packages/aiohttp/protocol.py", line 359, in parse_length_payload
out.feed_data(chunk, len(chunk))
File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 554, in feed_data
self._stream.transport.pause_reading()
File "/usr/lib64/python3.5/asyncio/sslproto.py", line 336, in pause_reading
self._ssl_protocol._transport.pause_reading()
File "/usr/lib64/python3.5/asyncio/selector_events.py", line 647, in pause_reading
raise RuntimeError('Cannot pause_reading() when closing')
RuntimeError: Cannot pause_reading() when closing
Steps to reproduce
The relevant code is something like this, though I've only been able to reproduce it in an application which creates many sessions simultaneously:
async def ssl_fetch(url):
with aiohttp.ClientSession(
connector=aiohttp.TCPConnector(
ssl_context=ssl._create_unverified_context())) as session:
resp = await session.get(url)
content = await resp.read()
await resp.release()
return content
Your environment
I have observed this bug with aiohttp versions 1.0.1 and 1.0.2 (have not tried any older versions), on Gentoo Linux with Python 3.5.2. I've only been able to reproduce it in an application which creates many sessions simultaneously. Each session has a single request directed at an https endpoint on port 5080 of Linux host running ganeti (http://docs.ganeti.org/ganeti/2.5/man/ganeti-rapi.html).
I was able to workaround the problem by overriding the ClientSession response_class parameter as follows:
class _FlowControlStreamReader(aiohttp.streams.StreamReader):
def feed_data(self, data, size=0):
super().feed_data(data)
class _ClientResponse(aiohttp.client_reqrep.ClientResponse):
# workaround bug in FlowControlStreamReader
flow_control_class = _FlowControlStreamReader
ivan-is commented
the same error
RuntimeError: Cannot pause_reading() when closing
with this code:
import asyncio
import aiohttp
async def crawler(loop):
url = 'http://www.powertoolsdirect.com/hand-tools/laser-levels'
async with aiohttp.ClientSession(loop=loop) as session:
async with session.get(url) as response:
print(response.status)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(crawler(loop))
loop.close()
environment:
pip3.5 freeze
aiohttp==1.2.0
async-timeout==1.1.0
chardet==2.3.0
yarl==0.8.1
`which python3.5` -V
Python 3.5.2
IaroslavR commented
Confirm this bug on Ubuntu 16.04 with Python 3.6.0b4 and aiohttp 1.2.0
Traceback:
File "/home/iaroslavr/projects/crawler.py", line 88, in _get_html
html = await response.text()
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 747, in text
yield from self.read()
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 718, in read
self._content = yield from self.content.read()
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 517, in wrapper
result = yield from func(self, *args, **kw)
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 582, in read
return (yield from super().read(n))
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 264, in read
raise self._exception
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/parsers.py", line 188, in set_parser
next(p)
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/protocol.py", line 305, in __call__
yield from self.parse_chunked_payload(out, buf)
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/protocol.py", line 348, in parse_chunked_payload
out.feed_data(chunk, len(chunk))
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/protocol.py", line 392, in feed_data
self.out.feed_data(chunk, len(chunk))
File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 573, in feed_data
self._stream.transport.pause_reading()
File "/usr/local/lib/python3.6.0b4/lib/python3.6/asyncio/sslproto.py", line 344, in pause_reading
self._ssl_protocol._transport.pause_reading()
File "/usr/local/lib/python3.6.0b4/lib/python3.6/asyncio/selector_events.py", line 702, in pause_reading
raise RuntimeError('Cannot pause_reading() when closing')
RuntimeError: Cannot pause_reading() when closing