john-kurkowski/tldextract

Exception reading Public Suffix List url

petroslamb opened this issue · 3 comments

petroslamb commented

Seems that the cache is not loaded for some reason?

I can see some json files deep under ~/.cache/python-tldextract have been created and I think I have whitelisted the right IP ranges to allow fetching TLD lists, the original error was Cannot download list of TLDs. so I am past that. Any idea what is causeing this?

Exception reading Public Suffix List url https://publicsuffix.org/list/public_suffix_list.dat
Traceback (most recent call last):
  File "/home/workable/app/.venv/lib/python3.11/site-packages/tldextract/cache.py", line 206, in run_and_cache
    result = cast(T, self.get(namespace=namespace, key=key_args))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/workable/app/.venv/lib/python3.11/site-packages/tldextract/cache.py", line 100, in get
    raise KeyError("namespace: " + namespace + " key: " + repr(key))
KeyError: "namespace: publicsuffix.org-tlds key: {'urls': ('https://publicsuffix.org/list/public_suffix_list.dat', 'https://raw.githubusercontent.com/publicsuffix/list/master/public_suffix_list.dat'), 'fallback_to_snapshot': True}"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/workable/app/.venv/lib/python3.11/site-packages/tldextract/cache.py", line 206, in run_and_cache
    result = cast(T, self.get(namespace=namespace, key=key_args))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/workable/app/.venv/lib/python3.11/site-packages/tldextract/cache.py", line 100, in get
    raise KeyError("namespace: " + namespace + " key: " + repr(key))
KeyError: "namespace: urls key: {'url': 'https://publicsuffix.org/list/public_suffix_list.dat'}"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/workable/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/workable/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/home/workable/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/workable/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/home/workable/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 653, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/workable/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 806, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
             … [message truncated due to size]

tldextract 3.5.0
Python 3.11.9

petroslamb commented

Looks somewhat related to #332

john-kurkowski commented

After your error, does the library function as expected with the latest public suffix definitions? Or do you get no results, the library's unusable?

The "Exception reading Public Suffix List url" message is trying the 1st of 2 default URLs for public suffix definitions. It's possible the 2nd URL succeeds and the library works as expected for you. (Still, I've reduced the "Exception reading Public Suffix List url" from an error to a warning in 921a825.)

… [message truncated due to size]

I wonder what that urllib3 exception is. It might indicate trouble reaching publicsuffix.org. It also might be a red herring, as your log seems to collect several benign exceptions that happened in recent time even if they were handled.

petroslamb commented

Yes the library is useable and I think I have managed to mitigate this error by whitelisting a wider range of ips. This error might also appear in cases of intermittent connectivity. Thanks for your answer on this.

  • 👍1