Handle Upcoming Tornado HTTPClient Errors

We’re getting a bunch of warnings from Tornado in our tests that future releases will raise some errors even if HTTPClient.fetch(raise_error=False):

/home/circleci/web-monitoring-processing/venv/lib/python3.6/site-packages/tornado/httpclient.py:294: DeprecationWarning: raise_error=False will allow 'Unsupported url scheme: example.org' to be raised in the future
  response.error, DeprecationWarning)

/home/circleci/web-monitoring-processing/venv/lib/python3.6/site-packages/tornado/httpclient.py:294: DeprecationWarning: raise_error=False will allow 'Unsupported url scheme: example.org' to be raised in the future
  response.error, DeprecationWarning)

/home/circleci/web-monitoring-processing/venv/lib/python3.6/site-packages/tornado/httpclient.py:294: DeprecationWarning: raise_error=False will allow '[Errno -2] Name or service not known' to be raised in the future
  response.error, DeprecationWarning)

/home/circleci/web-monitoring-processing/venv/lib/python3.6/site-packages/tornado/httpclient.py:294: DeprecationWarning: raise_error=False will allow '[Errno -2] Name or service not known' to be raised in the future
  response.error, DeprecationWarning)

Basically, it sounds like it will start only affecting whether an error is raised in response to > 400 status code, and other kinds of errors will always get raised.

Since we don’t wrap that in a try...except block, that’s going to cause problems, and we need to fix it.

Whoops, should have noted the code at issue is here, in diffing_server.DiffHandler.get():

web-monitoring-processing/web_monitoring/diffing_server.py

Lines 153 to 162 in 297b4a4

    
           fetched = yield [client.fetch(url, headers=headers, raise_error=False) 
        
                            for url in to_fetch.values()] 
        
           responses.update({param: response for param, response in 
        
                             zip(to_fetch, fetched)}) 
        
           try: 
        
               for response in responses.values(): 
        
                   self.check_response_for_error(response) 
        
           except tornado.httpclient.HTTPError: 
        
               return

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in seven days if no further activity occurs. If it should not be closed, please comment! Thank you for your contributions.

	fetched = yield [client.fetch(url, headers=headers, raise_error=False)
	for url in to_fetch.values()]
	responses.update({param: response for param, response in
	zip(to_fetch, fetched)})

	try:
	for response in responses.values():
	self.check_response_for_error(response)
	except tornado.httpclient.HTTPError:
	return