Running top test, with one nested failing test, causes all nested tests to be marked as failed
fredrikaverpil opened this issue · 2 comments
Expected behavior
When running a test with nested sub-tests, if one sub-tests fail, it is expected that only this sub-test is marked as failed along with the top-level test in the test summary and in the gutter.
It's possible that this issue lies in Neotest itself, and if so, it's an upstream issue.
Current behavior
Screenshot below, in which I ran the TestSumAllTails top-level test. The expected outcome should've been to mark only the more than one element sub-test and the TestSumAllTails top-level test.
Notes
If instead executing testing on the "file" or "folder" (in the screenshot below, I ran the folder lgwt), the tests are correctly marked:
Now that #32 was merged, it's possible that the population of statuses could be done in a similar manner as when running tests using the dir position.type.
There might be slight differences here when executing a single "top" test (with subtests) vs running a file of tests... but;
When running a "top" test, there is just one command to be executed. This command may have several passing subtests and all of a sudden one failing subtest. The result of the command is non-zero, and any subtest executing after this failing subtest will also be marked as failed.
Therefore it's a good idea not to rely on the command execution's status code for the subtests.