HackSoc/csbot

Use client PING to detect dead connections

alanbriolat opened this issue · 1 comments

If a server connection is lost in a way that doesn't explicitly close the TCP connection, the bot's side of the connection will remain open until TCP idle timeout, which is usually 2 hours. The solution is to have the client PING the server on a regular basis, so a dead connection actually dies when it fails to send.

/cc @LordAro

Also, when it does notice the connection is dead, it doesn't reconnect:

bot_1      | Traceback (most recent call last):
bot_1      |   File "/app/run_csbot.py", line 3, in <module>
bot_1      |     main(auto_envvar_prefix='CSBOT')
bot_1      |   File "/venv/lib/python3.6/site-packages/click/core.py", line 722, in __call__
bot_1      |     return self.main(*args, **kwargs)
bot_1      |   File "/venv/lib/python3.6/site-packages/click/core.py", line 697, in main
bot_1      |     rv = self.invoke(ctx)
bot_1      |   File "/venv/lib/python3.6/site-packages/click/core.py", line 895, in invoke
bot_1      |     return ctx.invoke(self.callback, **ctx.params)
bot_1      |   File "/venv/lib/python3.6/site-packages/click/core.py", line 535, in invoke
bot_1      |     return callback(*args, **kwargs)
bot_1      |   File "/app/csbot/__init__.py", line 96, in main
bot_1      |     client.loop.run_until_complete(client.run())
bot_1      |   File "/usr/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete
bot_1      |     return future.result()
bot_1      |   File "/app/csbot/irc.py", line 270, in run
bot_1      |     await self.read_loop()
bot_1      |   File "/app/csbot/irc.py", line 306, in read_loop
bot_1      |     line = await self.reader.readline()
bot_1      |   File "/usr/lib/python3.6/asyncio/streams.py", line 488, in readline
bot_1      |     line = yield from self.readuntil(sep)
bot_1      |   File "/usr/lib/python3.6/asyncio/streams.py", line 581, in readuntil
bot_1      |     yield from self._wait_for_data('readuntil')
bot_1      |   File "/usr/lib/python3.6/asyncio/streams.py", line 464, in _wait_for_data
bot_1      |     yield from self._waiter
bot_1      |   File "/usr/lib/python3.6/asyncio/selector_events.py", line 725, in _read_ready
bot_1      |     data = self._sock.recv(self.max_size)
bot_1      | ConnectionResetError: [Errno 104] Connection reset by peer