An example of how to generate code coverage reports from ros tests
To test this out, clone it into a workspace and then run
catkin test ros_coverage_example
You will see the results of the trivial tests that are implemented.
It's possible to generate coverage somewhat manually. Using the coverage package, we can add a launch prefix to the test launch files.
<launch>
<test test-name="my_code" pkg="ros_coverage_example" type="test_my_code_ros.py"
launch-prefix="coverage run -p"/>
</launch>With this launch prefix, when the tests are run you will see some .coverage prefixed files generated in the ~/.ros directory.
To generate the coverage, run the tests with
catkin test ros_coverage_example
If there is only one test, it may generate a single coverage file. If there are more, it's likely that they have suffixes to distinguish them, e.g.
michal@vetinari ~ $ ls -la ~/.ros/.cover*
-rw-r--r-- 1 michal michal 53248 Apr 19 15:14 /home/michal/.ros/.coverage.vetinari.115839.438412
-rw-r--r-- 1 michal michal 110592 Apr 19 15:14 /home/michal/.ros/.coverage.vetinari.115899.092796
We want to have these files combined so that we can look at the combined coverage of all of the tests.
To do that, we can use the coverage combine command, which will combine everything and generate a single .coverage file.
cd ~/.ros
coverage combine .coverage*
And the directory looks something like
michal@vetinari ~ $ ls -la ~/.ros/.cover*
-rw-r--r-- 1 michal michal 110592 Apr 19 15:16 /home/michal/.ros/.coverage
Once we have that coverage file, we can generate a report with
coverage report
However, without any filtering, this will usually show a lot of things that you're not interested in, like ros modules, numpy, and any other packages you've imported. To reduce the output we can use a filter like
coverage report --include="*src/ros_coverage_example*"
which will generate a much more readable report
Name Stmts Miss Cover
---------------------------------------------------------------------------------------------------------------
/home/michal/git/independent/ros_coverage_example/src/ros_coverage_example/my_code.py 29 8 72%
/home/michal/git/independent/ros_coverage_example/src/ros_coverage_example/my_code_ros.py 21 3 86%
---------------------------------------------------------------------------------------------------------------
TOTAL 50 11 78%
You can also use that coverage file to generate some html files to view the coverage more conveniently.
cd .ros
coverage html --include="*src/ros_coverage_example*"
xdg-open htmlcov/index.html
The code_coverage package is a useful tool which adds some cmake commands to generate coverage.
You can install it with
sudo apt install ros-noetic-code-coverage
There are instructions in that repository on how to install it, but you can also see the changes made in cmakelists and package xml in the commit ce1922 in this repository.
Key points:
- If using test launch files for python, you must add
launch-prefix="python3-coverage run -p"to the<test>tag. - Make sure you are using
add_rostestalong with test launch files to make sure that coverage is actually generated. Using onlycatkin_add_gtestis not sufficient for code_coverage to generate the coverage reports.
If using symlinked repositories, make sure you have the commit from the PR at mikeferguson/code_coverage#37. Without that, python coverage generation will not be fully automated.
When trying code_coverage initially, I was using coverage installed with pip3 install coverage. There seems to be some version mismatch or other issues even if python3-coverage is actually installed. I ended up making a symlink with
sudo ln -s /home/michal/.local/bin/coverage /usr/bin/python3-coverage
which made everything work.