After updating xUnit and xunit.runner.visualstudio versions, unit tests that calls DirectoryInfo.Delete fail with System.UnauthorizedAccessException

[SenseYang]

I have a repo opened in Visual Studio that refers to:

And after I updated both versions to 2.5.0, unit tests that calls DirectoryInfo.Delete API fail with exceptions like:

Message: 
System.UnauthorizedAccessException : Access to the path '<some file>.dll' is denied.

Stack Trace: 
FileSystem.RemoveDirectoryRecursive(String fullPath, WIN32_FIND_DATA& findData, Boolean topLevel)
FileSystem.RemoveDirectory(String fullPath, Boolean recursive)
DirectoryInfo.Delete(Boolean recursive)
...

But if I change the version to:

<PackageReference Include="xunit" Version="2.5.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" >

Then all previously failed unit tests will pass.

Note:

1. The directory that is to be deleted in the unit test will contain an assembly file as the build artifact from another unit test project, using PostBuildEvent build target like:

<Target Name="PostBuild" AfterTargets="PostBuildEvent">
    <Exec Command="copy /Y $(TargetDir)$(TargetName).* $(SolutionDir)UnitTestOutput\bin\$(ConfigurationName)\$(UnitTestProjectNameThatFailsAfterVersionBump)\$(TargetFramework)\" />
</Target>

2. The project uses .net6. Other dependencies are:

<PackageReference Include="coverlet.collector" Version="6.0.0" >
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
  <PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="FluentAssertions" Version="6.11.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.3" />
<PackageReference Include="NSubstitute" Version="5.0.0" />
<PackageReference Include="xunit" Version="2.4.5" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" >
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
  <PrivateAssets>all</PrivateAssets>
</PackageReference>

Please help look into this issue. Thanks!

[bradwilson]

@SenseYang The only call to Directory.Delete that could be used by xunit.runner.visualstudio is done against the temporary directory used for shadow copy:

https://github.com/xunit/xunit/blob/9ca64f18c28add3261dad902e46837e2bf1224b5/src/xunit.runner.utility/AppDomain/AppDomainManager_AppDomain.cs#L100

This code is not new in 2.5.0 (that code was added 10 years ago!), and the only files that should exist there are the ones that .NET itself copies over when doing the shadow copy in your app domain. And it's wrapped in a try/catch block to ensure that even if it fails, it shouldn't break anything.

There are no calls to Directory.Delete directly in xunit.runner.visualstudio anywhere.

There are no calls to DirectoryInfo.Delete anywhere in the xUnit.net source code. It's possible that Directory.Delete delegates to this, but you've trimmed the stack trace in question. In point of fact, I don't see anything from xUnit.net in that stack trace, so I don't know the provenance of the call at all.

In short, there's no way for me to reproduce this based on the information you've provided. A repro project would be best, but at the very least, more detailed information is required.

The project uses .net6

It's also worth mentioning: app domains and shadow copying are only used by .NET Framework. They're not used by .NET Core/.NET, so even the one potential line of code that might have been the culprit would never be called.

[bradwilson]

Closing for lack of response. Please open a new issue if this is still a problem. Thanks!