Why does coverage increase by 0.1% if i changed only the codecov.yml file?
Closed this issue · 2 comments
all i did is touching the codecov.yml file to disable patch
this is pretty unreliable
Where exactly does the coverage change? When there are higher-level, integration style tests present in a project, it usually happens that the set of executed lines during a CI run is nondeterministic, and as a consequence, the coverage changes by +/- few percent from one run to the next, seemingly by itself.
I saw this happening in some tests that were starting servers on localhost and connecting to them. Depending on timing, the code that was executed differed from run to run. This could be "fixed" by testing the networking code more thoroughly in isolation, within unit tests, and making sure that all possibilities, one of which always happened in integration tests, are covered by those unit tests.
One thing that can help is to exclude the test code itself from coverage measurement (but then you lose detection of dead test code).
@gresbtim, please open a ticket for this in support on our community boards.