rust-lang/rust

Provide indicator of doctest attributes with --persist-doctests

xd009642 opened this issue · 1 comments

Relevant tarpaulin issue: xd009642/tarpaulin#924

So in cargo-tarpaulin we use --persist-doctests in order to keep the doctest binaries around and run them for doctest coverage. In order to maintain the equivalent behaviour to cargo test, we need to keep track of the no_run and ignore attributes, as well as ensure the should_panic doctests exit with a non-zero exit code.

Currently the persisted doctests have a file name which is equivalent to the pah to the source file with the directory separator replaced with an underscore underscore separated integers for line number and also an index to prevent name collisions (likely due to the fact that src/foo/bar.rs and src/foo_bar.rs will normalise to same name when replacing directory separators).

The approach then taken is from the doctest name, it works out candidate files and locations, and then from this checks there for a doctest and looks for the attributes. However, this is just a heuristic and things like #![doc = include_str!("../README.md")] where the readme contains doctests using attributes cause the heuristic to fail.

Adding the attributes to the binary name would be a simple solution. Some way of querying metadata for a doctest and finding the binary name and tying it to a file location would also work. As far as I know there's no stability guarantee for the doc test binary naming - although I may be incorrect. Is there an openness to adapting the doc test binary name or metadata in order to make it simpler to take the persisted doctests and handle them the same way cargo test will?

I think the best way forward might be that --message-format=json exposes additional lines related to doctests, with file and line numbers.