Override error return code (blocking CI)
hugorodgerbrown opened this issue · 2 comments
If the codeclimate-test-reporter encounters an error unrelated to coverage - e.g. requests.exceptions.Timeout (https://github.com/codeclimate/python-test-reporter/blob/master/codeclimate_test_reporter/components/runner.py#L59) it will return a status code of 1, indicating an exception. This causes our CI (Codeship) to register a test run failure - in a specific case today we were getting 502 responses from the Codeclimate API - which was killing our builds, despite our test suite passing.
In this situation, our preference would be to fail on the exception by default (the current behaviour - so we know that there is a possible API issue), but then to be able to override this manually, for instance by setting an env var:
CODECLIMATE_FAIL_ON_ERROR=False
We could then set this env var when we know there is an issue, and then unset it once the issue has passed. Codeclimate is important for us, but not a deployment blocker - and since we have automated deployments running on a successful build, test run failures can have a significant impact.
@hugorodgerbrown I think it would be best for our reporter to continue to behave in the way that it does, which is a best practice for binaries like ours. Would it work for you to do something like codeclimate-test-reporter || true?
@gordondiggs good question - no idea. Will try that. Thanks for quick response.