edgi-govdata-archiving/web-monitoring-processing

DB client should set the `Accept` header on requests

Mr0grog opened this issue · 1 comments

Technically, the web-monitoring-db client in db.py should set the HTTP Accept header to application/json when making requests.

Most of the time it’s ok not to set that header because the server will almost always return JSON regardless of what you asked for, but not always! In edgi-govdata-archiving/web-monitoring-db#614, we’ll be using that header to determine what to do in dev mode, which will affect what type of exception you get when a request fails.

Right now, each method on db.Client is responsible for making its own HTTP request. For example:

def get_page(self, page_id):
"""
Lookup a specific Page by ID.
Parameters
----------
page_id : string
Returns
-------
response : dict
"""
url = f'{self._api_url}/pages/{page_id}'
res = requests.get(url, auth=self._auth)
_process_errors(res)
result = res.json()

We should probably pull the request, error handling, and JSON parsing into a separate private method. Then make sure that method sets the Accept header for every request.