hashgraph/hedera-sdk-js

Set codecov threshold and target

ivaylonikolov7 opened this issue · 1 comments

Problem

Currently codecov fails if the test coverage drops slightly. This is an issue as our tests seem to be non-deterministic and sometimes coverage drops non-signifcantly by -0.04%, -0.1%, 0.5% etc I couldn't investigate if this is an issue with codecov itself not following all test paths or there is some randomness generated in our tests too.

This creates the perception that our main branch is broken because the CI says it failed when it actually didn't.

Solution

My solution is that we should set a codecov threshold of 1% so even if our tests drop slightly the CI shouldn't have this ugly red cross.

But I would like to also set a target that the maintainers of SDK should aim for so this 1% threshold doesn't get abused and we don't fall significantly in our tests coverage. Currently we have approximately 84% coverage so a realistic starting target would be 80%. In my opinion we should aim long term for an 90% target but this might take some time to reach. I think that more than 90% coverage would mean that we will have bloated tests, duplication, time wasted and tests that are hard to read.

Alternatives

disable codecov <3

complete via #2575