Missing support for parallelized builds on failure
rogercampos opened this issue · 6 comments
When running your test suite parallelized across multiple process or even machines, it's possible for just one of those executions to fail. This should be considered as a global failure, but instead all the successful executions are correctly reported to codecov except for the failed one, resulting in an aberrant coverage report.
I understand this is difficult to accomplish due to the fact that the reporting is integrated as an rspec formatter directly, loosing control over all the executions. For this to work I assume a refactor should be done to separate the steps of collecting metrics and reporting, providing the user with granular control about when to execute the submission of data to codecov.
This should be considered as a global failure
Codecov does check CI build state. https://docs.codecov.io/docs/ci-service-relationship
Typically users will upload coverage from each container when they pass. Codecov will handle the overall commit status. Skipping commits that failed 👍
Oh! then I understand that codecov has specific code to talk with all those providers server-side and pull the status information. Custom CI runners are not supported, outside that list? Is there a generic api I can use to control this?
Or, better, can I have codecov check the status against the github status? https://developer.github.com/v3/repos/statuses/
Codecov does check GitHub API's Statuses 👍
Ok then, thanks for you help and time!
My pleasure. https://docs.codecov.io/docs/detecting-ci-services