toddrob99/searcharr

Slow bot

Closed this issue · 4 comments

Hello,

I've been running searcharr on my NAS without any flaws for the past couple of months, but have since had to format it and reinstall it. (Not entirely sure which version I had, sorry, I know it would have been useful)

Ever since the reformat, the bot is very slow for some reason and it seems like it's spamming the logs with the following error:

Error while getting Updates: urllib3 HTTPError HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot`<telegram-token-removed>`/getUpdates (Caused by ConnectTimeoutError(<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x7f70d09550d0>, 'Connection to api.telegram.org timed out. (connect timeout=5.0)'))

Here's some more logs when I boot up the docker container:

  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", line 140, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 614, in urlopen
    httplib_response = self._make_request(conn, method, url,
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 360, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 857, in _validate_conn
    super(HTTPSConnectionPool, self)._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 289, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", line 284, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", line 149, in _new_conn
    raise NewConnectionError(
telegram.vendor.ptb_urllib3.urllib3.exceptions.NewConnectionError: <telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4129fe32b0>: Failed to establish a new connection: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/telegram/utils/request.py", line 259, in _request_wrapper
    resp = self._con_pool.request(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/request.py", line 68, in request
    return self.request_encode_body(method, url, fields=fields,
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/request.py", line 148, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/poolmanager.py", line 244, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 691, in urlopen
    return self.urlopen(method, url, body, headers, retries,
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 691, in urlopen
    return self.urlopen(method, url, body, headers, retries,
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 691, in urlopen
    return self.urlopen(method, url, body, headers, retries,
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 665, in urlopen
    retries = retries.increment(method, url, error=e, _pool=self,
  File "/usr/local/lib/python3.9/site-packages/telegram/vendor/ptb_urllib3/urllib3/util/retry.py", line 376, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
telegram.vendor.ptb_urllib3.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot<replaced>/getMe (Caused by NewConnectionError('<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4129fe32b0>: Failed to establish a new connection: [Errno -3] Try again'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/searcharr.py", line 1769, in <module>
    tgr.run()
  File "/app/searcharr.py", line 1435, in run
    updater.start_polling()
  File "/usr/local/lib/python3.9/site-packages/telegram/ext/updater.py", line 422, in start_polling
    self._init_thread(self.dispatcher.start, "dispatcher", ready=dispatcher_ready)
  File "/usr/local/lib/python3.9/site-packages/telegram/ext/updater.py", line 342, in _init_thread
    name=f"Bot:{self.bot.id}:{name}",
  File "/usr/local/lib/python3.9/site-packages/telegram/bot.py", line 369, in id
    return self.bot.id
  File "/usr/local/lib/python3.9/site-packages/telegram/bot.py", line 363, in bot
    self._bot = self.get_me()
  File "/usr/local/lib/python3.9/site-packages/telegram/bot.py", line 130, in decorator
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/telegram/bot.py", line 450, in get_me
    result = self._post('getMe', timeout=timeout, api_kwargs=api_kwargs)
  File "/usr/local/lib/python3.9/site-packages/telegram/bot.py", line 295, in _post
    return self.request.post(
  File "/usr/local/lib/python3.9/site-packages/telegram/utils/request.py", line 361, in post
    result = self._request_wrapper(
  File "/usr/local/lib/python3.9/site-packages/telegram/utils/request.py", line 265, in _request_wrapper
    raise NetworkError(f'urllib3 HTTPError {error}') from error
telegram.error.NetworkError: urllib3 HTTPError HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot<replaced>/getMe (Caused by NewConnectionError('<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4129fe32b0>: Failed to establish a new connection: [Errno -3] Try again'))

And my docker yml:

version: "3"
services:
  searcharr:
    image: toddrob/searcharr:v2
    container_name: searcharr
    environment:
      - PUID=1000
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /container-data/searcharr/logs:/app/logs
      - /container-data/searcharr/data:/app/data
      - /container-data/searcharr/settings.py:/app/settings.py
    restart: unless-stopped

The logs indicate a connectivity issue with the Telegram API. I can't say for sure why that is happening... Do you have any issues using the Telegram app on the same network?

I noticed you are specifying the v2 tag in your docker-compose.yml. I think there was a bug in my automated docker build scripts, because I didn't expect v2 to be a valid tag. If your Searcharr log says Searcharr v2.1.1 - Logging started! then you were on the latest version.

I just fixed the automated build scripts, and now the v2 tag is definitely the latest version (same as v2.1.3 and latest tags). Going forward, the v2 tag will be a way to get the latest version in until v3 comes around; or you can just use latest to always have the latest version.

hey @toddrob99 I think you're right - there seems to be an issue with the networking inside my docker containers, but don't understand why this is happening for some requests and not others.

I'm going to go ahead and close this issue 😄

@toddrob99 now that the networking issues are solved on the host, the bot slowness issues have been resolved.

thank you!