Provide indicator of doctest attributes with --persist-doctests
xd009642 opened this issue · 1 comments
Relevant tarpaulin issue: xd009642/tarpaulin#924
So in cargo-tarpaulin we use --persist-doctests in order to keep the doctest binaries around and run them for doctest coverage. In order to maintain the equivalent behaviour to cargo test, we need to keep track of the no_run
and ignore
attributes, as well as ensure the should_panic
doctests exit with a non-zero exit code.
Currently the persisted doctests have a file name which is equivalent to the pah to the source file with the directory separator replaced with an underscore underscore separated integers for line number and also an index to prevent name collisions (likely due to the fact that src/foo/bar.rs
and src/foo_bar.rs
will normalise to same name when replacing directory separators).
The approach then taken is from the doctest name, it works out candidate files and locations, and then from this checks there for a doctest and looks for the attributes. However, this is just a heuristic and things like #![doc = include_str!("../README.md")]
where the readme contains doctests using attributes cause the heuristic to fail.
Adding the attributes to the binary name would be a simple solution. Some way of querying metadata for a doctest and finding the binary name and tying it to a file location would also work. As far as I know there's no stability guarantee for the doc test binary naming - although I may be incorrect. Is there an openness to adapting the doc test binary name or metadata in order to make it simpler to take the persisted doctests and handle them the same way cargo test will?
I think the best way forward might be that --message-format=json
exposes additional lines related to doctests, with file and line numbers.