InvalidBufferError (HTTP 404), Growing TCP Connections

Question

InvalidBufferError (HTTP 404), Growing TCP Connections

Opened this issue 2 months ago · 1 comments

Code that causes the issue

2024-10-28 22:47:31,183 - ERROR - [telethon.network.mtprotosender:mtprotosender:396] - Unexpected exception reconnecting on attempt 2
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/python_socks/async_/asyncio/_proxy.py", line 62, in connect
    return await self._connect(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/python_socks/async_/asyncio/_proxy.py", line 73, in _connect
    _socket = await connect_tcp(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/python_socks/async_/asyncio/_connect.py", line 28, in connect_tcp
    await loop.sock_connect(sock=sock, address=address)
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 638, in sock_connect
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/python_socks/async_/asyncio/_proxy.py", line 61, in connect
    async with async_timeout.timeout(timeout):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 115, in __aexit__
    raise TimeoutError from exc_val
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/telethon/network/mtprotosender.py", line 378, in _reconnect
    await self._connect()
  File "/usr/local/lib/python3.11/site-packages/telethon/network/mtprotosender.py", line 234, in _connect
    connected = await self._try_connect(attempt)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/telethon/network/mtprotosender.py", line 284, in _try_connect
    await self._connection.connect(timeout=self._connect_timeout)
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/connection.py", line 250, in connect
    await self._connect(timeout=timeout, ssl=ssl)
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/connection.py", line 231, in _connect
    sock = await self._proxy_connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/connection.py", line 141, in _proxy_connect
    sock = await proxy.connect(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/python_socks/async_/asyncio/_proxy.py", line 68, in connect
    raise ProxyTimeoutError(f'Proxy connection timed out: {timeout}') from e
python_socks._errors.ProxyTimeoutError: Proxy connection timed out: 60

2024-10-28 23:47:28,751 - ERROR - [telethon.network.mtprotosender:mtprotosender:519] - Server sent invalid buffer
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/telethon/network/mtprotosender.py", line 507, in _recv_loop
    body = await self._connection.recv()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/connection.py", line 310, in recv
    raise err
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/connection.py", line 341, in _recv_loop
    data = await self._recv()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/connection.py", line 380, in _recv
    return await self._codec.read_packet(self._reader)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/telethon/network/connection/tcpfull.py", line 32, in read_packet
    raise InvalidBufferError(body)
telethon.errors.common.InvalidBufferError: Invalid response buffer (HTTP code 404)

2024-10-29 00:06:07,459 - WARNING - [telethon.network.connection.connection:connection:345] - Server closed the connection: 4 bytes read on a total of 8 expected bytes

Expected behavior

The application should not be stuck and be able to handle continuous use without accumulating TCP connections or encountering frequent reconnection cycles due to InvalidBufferError.

Actual behavior

After running continuously for a certain period, application experiences an increasing number of errors and tcp connection also in thid periond

Observed Behavior
Observe a spike in logs:
Repeated messages like: Assigned msg_id = to GetUsersRequest () and Closing current connection to begin reconnect...
eventually if you don't restart the app there will be errors msg too old and app will be stuck

Observed Patterns
Before InvalidBufferError was Proxy connection timed out
Frequent errors with InvalidBufferError, leading to repeated reconnection attempts.
Numerous warnings in the logs indicating Server closed the connection.
Increasing TCP connections without any specific high-demand triggers on the application side.
Increasing frequency of the log entries with the message Assigned msg_id = to GetUsersRequest

Traceback

No response

Telethon version

1.37.0

Python version

3.11.9

Operating system (including distribution name and version)

ubuntu 22, 5.15.0-122-generic, Docker with host mode network

Other details

Probably related issues: #3854 #4042

Checklist

The error is in the library's code, and not in my own.
I have searched for this issue before posting it and there isn't an open duplicate.
I ran pip install -U https://github.com/LonamiWebs/Telethon/archive/v1.zip and triggered the bug in the latest version.

Answer 1 · 2024-10-29T07:37:46.000Z

I appreciate the detailed report. Unfortunately I don't think I'll be able to spend much time investigating and fixing issues like these in v1.