Failed to load extensions from file NUnit3.TestAdapter.dll error in Azure DevOps Release pipeline ONLY
johnjoncyn opened this issue · 5 comments
I have created a vs 2022 project. The tests run sucessfully in VS2022 and Azure DevOps Build Pipeline.
The same fails when i run it in the release pipeline with the output of a sucessful build.
Here are the details
Microsoft Visual Studio Professional 2022 (64-bit) - Current Version 17.8.6
Project
net8.0</TargetFramework
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0"
<PackageReference Include="Microsoft.TestPlatform.TestHost" Version="17.9.0"
<PackageReference Include="NUnit" Version="4.0.1"
<PackageReference Include="NUnit3TestAdapter" Version="4.5.0"
Extract from Azure DevOps Release pipeline
2024-02-14T11:46:45.6204508Z Discovering tests from sources
2024-02-14T11:46:45.6266833Z Using new test platform for test execution
2024-02-14T11:46:45.6391145Z Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleWrapper instance will be created
2024-02-14T11:46:46.9661724Z ##[warning]DiscoveryMessage : Failed to load extensions from file 'C:\agent_work\r1\a_Google\drop\Google\NUnit3.TestAdapter.dll'. Please use /diag for more information.
2024-02-14T11:46:47.2890886Z DiscoveryMessage : NUnit Adapter 4.5.0.0: Test discovery starting
2024-02-14T11:46:47.4970377Z DiscoveryMessage : NUnit Adapter 4.5.0.0: Test discovery complete
2024-02-14T11:46:47.6706399Z Number of testcases discovered : 2
2024-02-14T11:46:47.6845103Z Discovered tests 2 from sources
2024-02-14T11:46:47.6845440Z =================================================================
2024-02-14T11:46:47.8186475Z [RunStatistics]This execution slice with id '15', received '2' testcases to execute out of which '2' is discovered.
2024-02-14T11:46:47.8239827Z **************** Starting test execution *********************
2024-02-14T11:46:47.8369647Z Using new test platform for test execution
2024-02-14T11:46:47.8370325Z Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleWrapper instance will be created
2024-02-14T11:46:48.6727182Z ##[warning]RunMessage : Failed to load extensions from file 'C:\agent_work\r1\a_Google\drop\Google\NUnit3.TestAdapter.dll'. Please use /diag for more information.
2024-02-14T11:46:49.1698511Z RunMessage : NUnit Adapter 4.5.0.0: Test execution started
2024-02-14T11:46:49.1877406Z RunMessage : Running selected tests in C:\agent_work\r1\a_Google\drop\Google\googling.dll
2024-02-14T11:46:49.2909656Z RunMessage : NUnit3TestExecutor discovered 2 of 2 NUnit test cases using Current Discovery mode, Non-Explicit run
2024-02-14T11:46:49.4417102Z RunMessage : NUnit Adapter 4.5.0.0: Test execution complete
Attached the release pipeline
I have tried
- Clean the build and rebuild the project
- add retore of nuget as one of the task on release pipeline
There is no extensions to the NUnit3Testadapter, so it doesn't make sense that it asks for that.
Notices that you reference the Microsoft.TestPlatform.TestHost, that should not be included.
Can you upload the yml file you're using ?
And the csproj for your project.
If you're using the old classic build you should convert to yml.
I see a message there refer to "new test platform". If that is the completely new test platform that was announced a few days ago, that should be turned off.
@OsirisTerje Thanks for the response. I have attached the yml and proj file for the build pipline where the automated Test Cases run sucessfully with no errors.
files.zip
Where would i find the details of config for the test platform.
In the release pipeline it dosent work as expected and i havent used yml to configure the release pipeline. please see the screenshot in the above description of what the release pipeline looks like.
The yml and csproj looks good.
The Azure DevOps release pipeline doesn't use YML files (afaik) , and is based off the old classic model. I am not sure what exactly it uses these days, and I don't think it is much in use for new systems today.
It might use the old tests tasks using VSTest.console, or some derivative of that. I am not sure at all here.
The way people do today is to add releases into the build pipeline, using yml files to describe them. That would work for you too as the current build pipeline works. There are a lot of information available on how to deploy using the build pipeline.
If you want to debug the release pipeline further, you need to come up with more information on what exactly goes wrong. Which task, and what is its configuration. With the classic model, it would be a ton of screenshots. As said above, the adapter doesnt have extensions, so it is not likely to create the crash itself. It must be some other component trying to do something it should not even try to do.
I would also consider why you would run tests in the release pipeline at all. They belong more in a build pipeline. The release is about deploying stuff somewhere.
@OsirisTerje That right. I am using the classic version. The objective is to use the release pipeline to update the Test Plan module in Azure devops. Test run in the test plan cant be updated with the test results from the Build.
Its very basic setup using an agent configured to run on my windows 10 and your right i an using visual studio test. The screenshot above shows the config. whats suprising is that it works in the build pipeline but not release.
Your build pipeline is running dotnet test and not vstest.console. I assume that the release pipeline is using the vstest.console, and there might be a difference there.
You can try locally to run either dotnet vstest or even vstest.console and see if that repro the crash.
I can't see why the build pipeline should not be able to update the Test Plan module. It has an API like everything else there.
UPDATE:
If you look into https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/VsTestConsoleWrapper.cs you can see this is the test console wrapper, and it has an extension mechanism (see line 49). It is not the new system, as I suspected earlier. So this is what is crashing. It might be wise to create an issue in that repo.
It should also be possible to repro this locally as I mentioned above.
You can also try to drop down to .net 6 and an earlier version of the SDK to see if that makes any difference here.