Test Coverage
AzureMarker opened this issue · 6 comments
We currently lack insight into test coverage. We need to look into test coverage tools so we can measure how well the code is tested.
The recommended service to use is Codecov (example docs). It can be supplied data from kcov (see the example docs) or tarpaulin.
Supports #30
Tested tarpaulin on development. It fails the return_pthread_unlock_errors test in shared_lock.rs. On further inspection it seems to be an issue with tarpaulin, possibly related to xd009642/tarpaulin#190. After making some debugging changes to the test (printing out the unlock_result variable), it segfaults when run through tarpaulin.
Also tested kcov. It was a little harder to set up (more dependencies), but it didn't crash when I ran it.
On the bright side, our code coverage is somewhere around 82-83% (83.75% according to tarpaulin, 81.9% according to kcov).
I ran kcov through cargo-kcov, which provides an easy way to install and run kcov.
We may run into issues like the ones detailed here: interledger/interledger-rs#78
This is yet to be seen.
The IntelliJ Rust plugin recently added code coverage support (still in progress for CLion):
https://intellij-rust.github.io/2019/07/22/changelog-102.html
It uses grcov, which we may also be able to use.
When counting just lines, code coverage is about 84%. However, when counting branches such as those caused by the ? operator, code coverage is more around 69%. It may be hard to add coverage for some of these failure cases, especially when the failures are caused by writing to a file (ex. generating dnsmasq file).