NUnitFixtureOrdering does not work since test Adapter 4.0.0+

[patlgrc]

With NUnit3TestAdapter 4.1.0 and 4.0.0, the NUnitFixtureOrdering does not work.

========== Starting test run ==========
NUnit Adapter 4.1.0.0: Test execution started
Running selected tests in C:\Demo\Fixtures\Fixtures\bin\Debug\net6.0-windows\Fixtures.dll
Exception NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryException, Exception thrown executing tests in C:\Demo\Fixtures\Fixtures\bin\Debug\net6.0-windows\Fixtures.dll
Expected ParameterizedMethod, Theory or GenericMethod, but was TestFixture
at NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryConverter.ExtractParameterizedMethodsAndTheories(NUnitDiscoveryTestFixture tf, XElement node) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\DiscoveryConverter.cs:line 299
at NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryConverter.ExtractAllFixtures(NUnitDiscoveryCanHaveTestFixture parent, XElement node) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\DiscoveryConverter.cs:line 214
at NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryConverter.ExtractTestFixtures(NUnitDiscoveryCanHaveTestFixture parent, XElement node) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\DiscoveryConverter.cs:line 267
at NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryConverter.ExtractAllFixtures(NUnitDiscoveryCanHaveTestFixture parent, XElement node) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\DiscoveryConverter.cs:line 229
at NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryConverter.ConvertXml(NUnitResults discovery) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\DiscoveryConverter.cs:line 185
at NUnit.VisualStudio.TestAdapter.NUnitEngine.DiscoveryConverter.Convert(NUnitResults discoveryResults, String assemblyPath) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\DiscoveryConverter.cs:line 137
at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 279
NUnit Adapter 4.1.0.0: Test execution complete
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 154 ms ==========

It is very annoying because this library is SO great and used by all our functional tests.

Linked to nunit3-vs-adapter/issues/923

nunit/nunit3-vs-adapter#923

repo :

https://github.com/PatrickLeGarrec/NUnitFixtureOrdering

[Sebazzz]

Based on this comment:

This is not unexpected, since it introduces a new type of fixture, and there is no object model for that to deserialize the message into. The current discovery mode uses a strongly typed object model. Now, the Legacy discovery mode should be able to handle this, but it fails too, not even sure why that is. And 3.17 fails too, so.... It might work with the nuget package though, I haven't tried that. And I assume you use the source to be able to use Net6.

I suspect this is unfixable, without forking the VS test adapter also.

[PatrickLeGarrec]

I found a new way to order the fixtures with Nunit :

class PlayList
{
public enum Fixtures // Order of execution
{
MyFirstFixture,
MySecondFixture
};
}

class BrowserArgs : IEnumerable
{
public IEnumerator GetEnumerator()
{
yield return new object[] { "Edge" };
yield return new object[] { "Chrome" };
yield return new object[] { "Firefox" };
}
}

NUNIT :

[TestFixtureSource(typeof(BrowserArgs)), Order((int)PlayList.Fixtures.MyFirstFixture)]
public class MyFirstFixture
{ ... }

[TestFixtureSource(typeof(BrowserArgs)), Order((int)PlayList.Fixtures.MySecondFixture)]
public class MySecondFixture
{ ... }