Split formatter and uploader

[williampollet]

Hello,

We are using Codecov with codecov-ruby. Would it be possible to split the format job and the uploader job?

It would allow the team to trust more the codecov reports, particularly when the specs fail. Right now when they fail, an incomplete report is sent to codecov, and it displays an alarming message saying that the coverage is only 11%, which is false. Thus, people not totally invested into it tend to ignore the message, because they doubt its reliability.

Furthermore, it would allow Rails user to use codecov with more flexibility: use the bash command uploader in a separate job of the circle CI workflow for instance, after having run the specs. Send the report conditionally (for instance when spec fail, or put a SKIP_CODECOV flag).

Thanks !

[asbjornu]

Splitting the formatter and uploader would also make it clearer exactly what has gone wrong. When specs fail it's now easy to misinterpret it as Codecov failing since both are done in the same step. Separating formatting and uploading would at least make it apparent when Codecov is at fault and not.

I also think that from a security perspective, it makes sense to not expose CODECOV_TOKEN to any other script or process than the uploader, since it's the only one that actually needs it. With the current implementation, all specs, SimpleCov, and potentially any Ruby Gem used in solution have access to the token. Now it's not the end of the world if a rogue Ruby Gem is able to tamper with my code coverage statistics, but in principle I think separation is good security practice.

Additionally, the separation would make it both possible and preferable to use the Codecov GitHub Action in certain circumstances.

[asbjornu]

It seems like a split between coverage gathering and uploading is possible by doing something like in this gist by @stephanvd.

[thomasrockhu]

This has been resolved in 0.4.2

[thomasrockhu]

The underlying bash uploader has the -z argument which will allow uploading the formatted report. We will release a new action this week or next for the Codecov Action