Issafalcon/neotest-dotnet

xUnit - Support for `ClassData` parameterized tests

Issafalcon opened this issue · 1 comments

A new treesitter query will be needed to capture the ClassData attribute when discovering tests.

The existing mechanism for finding the parameters for tests will need to be adjusted, and the only way of extrapolating what the enumerated values of the class are during test discovery, is to use the dotnet test -t command to list all the tests.

This command allows us to:

  1. Discover all tests
  2. Discover the FQN for all tests including the framework specific parameters (even for non-inlined data like with ClassData)
  3. Keep the existing order of the parameterized tests as they appear in the attribute list on the test method

The potential solution for this would be to use ts-query for the inline and non-inlined parameterized tests, then run an async process to fetch the list of parameterized tests, matching them to the base-name of the "parent" test node.

Adding these nodes to the discovered positions can be done in the same way as currently, just incrementing the "row" value in the ranges by one for each new child test.

If this works, it will be a framework agnostic mechanism, with the only differences being the queries used to discover the parameterized test attributes

NOTE: The downside would be that, for each test file with parameterized tests, we are potentially running the command for each file.

  • It would be ideal to run the process once in the solution or project root when files are being discovered for the first time, save the output of the process, and use this output when discovering parameterized tests

Despite best efforts the dotnet test -t command is lacking in the following regards:

  1. Only for xUnit does the fully qualified test name get output
  2. No options exist to distinguish a test with the same or similar "base" name from another, in other frameworks (NUnit or MSTest)

From the above limitations, the adapter is stuck when it comes to reliably listing and building the positions for parameterized tests when they are not inlined. As such, the existing mechanism is still the best compromise of reliability and flexibility for the adapter (in-lined tests only show up as individual tests in the explorer, with non-inlined tests being grouped together as a single listing, and all run together as one)