Automatically retrying coverage uploading
Mange opened this issue · 7 comments
We're having issues where coverage reports are sometimes not uploaded in some of our test splits, making the coverage graphs unstable and unreliable. It almost always looks like a temporary network issue, or server issue, since the other test splits upload successfully.
Is it possible to make the upload retry n times before giving up?
Also having this issue. It's compounded by the fact that codecov silently swallows the exception raised when it occurs. As a result, we end up in a state where all the test jobs have passed on CI (we run multiple test jobs in parallel), but the build fails due to a drop in code coverage, which looks like new functionality has been added without appropriate coverage, or coverage has been removed.
I'd sooner have the test runner exit non-zero so that CI job could be retried.
@stevepeak would you accept a PR that raised instead of swallowing that exception? We could make it opt-in if you prefer for backwards compatibility, although IMO it would be better to make more noise about the problem by default.
Looks like this is solved by #48. Did anyone have luck solving this problem using the changes from that PR?
#48 looks sensible, I haven't tried it though.
My main concern is not re-raising the failure. I'm hoping to put together a PR for that sometime.
@thomasrockhu Looks like #48 ended up being merged into #59 recently, any plans on a release that includes this?
@rohitpaulk, this should have been released as part of 0.1.17
Ah, nice - thanks @thomasrockhu!