avajs/ava

Spurious "previous failures in test files that were not rerun" in watch mode

mildmojo opened this issue · 2 comments

Actual

When AVA v6.1.0 runs in watch mode, it prints something like 4 previous failures in test files that were not rerun on every rerun, counting all previously-failed tests from all previous runs, even after re-running all tests with 100% passing results.

Expected

When AVA runs in watch mode, it should not report previous test failures. When all tests are passing, it should report that all tests are passing, full stop.

Reproduction

Minimal reproduction steps below, with annotations about observed behavior:

$ mkdir /tmp/ava-repro && cd /tmp/ava-repro && npm install --dev ava
$ npx ava --version
6.1.0
$ echo "require('ava')('repro', t => t.fail());" > test.js
$ npx ava -w

  ✘ [fail]: test › repro Test failed via `t.fail()`
  ─

  test › repro

  test.js:1

   1: require('ava')('repro', t => t.fail());
   2:                                        

  Test failed via `t.fail()`

  › test.js:1:32

  ─

  1 test failed [16:17:04]

Touch test.js to let AVA re-run the test:

 ✘ [fail]: test › repro Test failed via `t.fail()`
  ─

  test › repro

  test.js:1

   1: require('ava')('repro', t => t.fail());
   2:                                        

  Test failed via `t.fail()`

  › test.js:1:32

 ─

  1 test failed [16:39:56]
  1 previous failure in test files that were not rerun

Change t.fail() to t.pass() and save:

 ✔ test › repro
  ─

  1 test passed [16:19:00]
  2 previous failures in test files that were not rerun

Type 'r' in the test console and hit enter to re-run tests:

  ✔ test › repro
  ─

  1 test passed [16:19:51]
  2 previous failures in test files that were not rerun

The "previous failures" message never seems to clear. I can't figure out what it means or how it would be useful; all test files were rerun, and none had failures.

  1. Why does it say test files were not rerun, even when there's only one test file that was definitely rerun?
  2. Why does the "previous failures" count persist, going up every time a test fails during a run, even when there's only a single test in the suite?

There are some previous reports of similar issues (#2821, #2069, #2040), but they've all been closed, and this still seems to happen in the current release.

Thanks for the great bug report @mildmojo!

See #3297, the counters were never cleared correctly, so on every run they would increase.

Previous reports were likely closed because the watch mode was rewritten, and that should have resolved any remaining issues, but for a path inconsistency.

Thanks for the fast fix! 🙌