Lambda Timeout connecting to CourtListener (HTTPSConnectionPool(host='www.courtlistener.com', port=443))

[sentry-io]

Sentry Issue: RECAPEMAIL-LAMBDA-2

timeout: The read operation timed out
(4 additional frame(s) were not displayed)
...
  File "http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)

ReadTimeoutError: HTTPSConnectionPool(host='www.courtlistener.com', port=443): Read timed out. (read timeout=5)
(2 additional frame(s) were not displayed)
...
  File "urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "urllib3/packages/six.py", line 770, in reraise
    raise value
  File "urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "urllib3/connectionpool.py", line 451, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "urllib3/connectionpool.py", line 340, in _raise_timeout
    raise ReadTimeoutError(

ReadTimeout: HTTPSConnectionPool(host='www.courtlistener.com', port=443): Read timed out. (read timeout=5)
(3 additional frame(s) were not displayed)
...
  File "requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "requests/adapters.py", line 578, in send
    raise ReadTimeout(e, request=request)

[albertisfu]

In #10 we added a timeout of 5 seconds for each request made to CL. In addition, we try every request 3 times with a backlog of 3 and a delay of 5 seconds so on average the lifetime of each request before giving up is around 1 minute.

Since the date #10 was released (October 13) we have received 2623 emails and received 6 timeouts from CL, so the failure rate is low, but we could reduce it even more maybe increasing the number of retries and the delay between each retry.

These failures might have happened while CL was unavailable for more than one minute, so we could increase time by 3 or 5 minutes before giving up retries so the possibility of delivering the message is greater.

[mlissner]

That sounds like a good plan to me. Seems like turning up the retries should avoid further problems.