feat request: Markdown format
Opened this issue · 1 comments
Summary
Github actions recently added the ability to output a Markdown-formatted test summary :
https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/
It would be great to have a straightforward Markdown output mode that would populate the test outcomes as a Markdown table.
Details
The nextest output is awesome for the CLI (seriously, thanks!), but contains some duplicate formatting for tests marked as slow, and is close to (but not quite a) Markdown table. For a (biased) example:
PASS [ 43.808s] arecibo tests::test_ivc_nondet_with_compression
SLOW [> 60.000s] arecibo supernova::snark::test::test_compression_with_circuit_size_difference
SLOW [> 60.000s] arecibo supernova::snark::test::test_nivc_trivial_with_compression
SLOW [> 60.000s] arecibo tests::test_ivc_nontrivial_with_compression
SLOW [> 60.000s] arecibo tests::test_ivc_nontrivial_with_spark_compression
SLOW [> 60.000s] arecibo tests::test_pp_digest
PASS [ 81.895s] arecibo tests::test_ivc_nontrivial_with_compression
PASS [ 81.349s] arecibo tests::test_pp_digest
SLOW [>120.000s] arecibo supernova::snark::test::test_compression_with_circuit_size_difference
SLOW [>120.000s] arecibo supernova::snark::test::test_nivc_trivial_with_compression
SLOW [>120.000s] arecibo tests::test_ivc_nontrivial_with_spark_compression
PASS [ 138.131s] arecibo supernova::snark::test::test_nivc_trivial_with_compression
PASS [ 136.331s] arecibo tests::test_ivc_nontrivial_with_spark_compression
PASS [ 148.281s] arecibo supernova::snark::test::test_compression_with_circuit_size_difference
Other approaches / relevant resources
It's hard to find public repos using nextest in conjunction with $GITHUB_STEP_SUMMARY
today.
It may be possible to scrounge up something out of the tool from the following (incomplete, not rigorously tested, doesn't cover ignored tests):
# Extract lines, then use awk to format as markdown table
grep -E "PASS|FAIL" | awk 'BEGIN {
print "| status | runtime | test |";
print "| --- | --- | --- |";
}
{
match($0, /(PASS|FAIL) \[\s*([^\]]+)\]( .*?)/, groups);
status = groups[1];
runtime = groups[2];
test = groups[3]
gsub(/^ /, "", test);
print "| " status " | " runtime " | " test " |";
}'
... or something even more rigorous from the json output.
But this issue is written in the hope contributors might find a maintained markdown-summary output mode a fun endeavor (and that the maintainers would find it a valuable contribution), based on the GH action use case.
By comparison, there is the cargo pretty-test
tool, which contains fixed-width output suitable for display with $GITHUB_STEP_SUMAMRY
(as text).
See also #20
Hi François, hope you're well! I agree that this would be great and I think nextest should learn how to write a markdown summary to a file.