Check that responses have correct status
Opened this issue · 0 comments
chrismytton commented
Problem
If a response fails and returns a non-200 error we currently don't do anything about it and pass the response untouched into the scraper class. It would be useful if Scraped did some kind of error checking on the status code.
Proposed solution
By default, we should check for a 200 response and then either print a warning or raise an exception if there is a non-200 response. This should be configurable so that a class can change the expected status, perhaps with a class level expected_status 201
type macro.