nextest-rs/nextest

Hide `PASS` tests/Show only `FAIL` tests

Opened this issue · 5 comments

Astlaan commented

For me, the output of cargo nextest is sometimes a bit too verbose. Sometimes I would prefer not to see all the "PASS" messages, but only the FAILED ones. This would be handy especially in a project with many tests, as it would make for a lot more concise output. There's not as much need to know which tests are passing, compared to which ones are failing. Is this possible?

sunshowers commented

Hi there — this is possible with the --status-level option. I'd recommend setting it to --status-level slow.

See https://nexte.st/book/other-options.html#--status-level-and---final-status-level.

Astlaan commented

Amazing, that's indeed what I was looking for.
I think it would be perhaps nice if "skip" was a separated status level though.

This way you could tell nextest to only display failing tests and skipped tests. Failing because these matter the most, and skipped so you don't forget your ignored tests. Displaying skipped is more useful than displaying passed, but skipped implies passed currently.

sunshowers commented

Interesting, hadn't thought of it that way! Will think through the implications of that. I think that passed is generally more important than skipped, but skipped + failed makes a ton of sense.

  • 👍1
Astlaan commented

I think that passed is generally more important than skipped

Interesting. My reasoning was the following:

  • If a test is passing, I suppose it is fine and doesn't need to be touched.
  • If it is failing, it needs to be solved, and for that you need to know which one is failing.
  • If it was skipped/ignored, then it was a problem that was temporarily swept under the rug and needs to be fixed at some point. Being displayed is then important, because it lets you know the problem is (still) there, and also lets you know what specific test was skipped, so you can fix it.

Hence why I thought that maybe skipped could have higher priority than failed.

sunshowers commented

If you run just one test on the CLI, though, you don't want to see the entire list of other tests as skipped. But I definitely see the argument for failed + skipped, and I think I have a design in mind to support both cases.