Drop-in replacement for requests
with default timeouts and retry support
requests
has no timeout by default and can hang forever psf/requests#3070requests
has poor retry support. It doesn't retry transient errors by default, and it's buried under "Lower-Lower-Level Classes" https://requests.readthedocs.io/en/v2.9.1/api/#lower-level-classes https://requests.readthedocs.io/en/v2.9.1/api/#requests.adapters.HTTPAdapter
rerequests
only changes convenience usages (e.g. requests.{get,put,...}):
- Sensible timeouts by default
- Conveniently configurable
max_retries
andRetry()
- Retry by default implements truncated exponential backoff with jitter
And provides the following extensions:
- Convenient requests.http_raise hook for 4XX and 5XX exceptions
Examples:
>>> import utils.rerequests as requests # drop-in replacement
>>> # as normal, but with timeouts by default (and no retries)
>>> requests.get(URL)
>>> requests.get(URL, timeout=(3.05, 27)) # override default
>>> # accessible retry config
>>> requests.post(URL, max_retries=10) # retry transient errors despite POST
>>> requests.put(URL, max_retries=requests.Retry(5, **more_config))
>>> # automatically response.raise_for_status(), with logging
>>> requests.delete(URL, hooks=requests.http_raise)
Resources: