Can not embed Microsoft.Data.SqlClient 2.1.1 with Costura 5.0.0-beta0001
0xced opened this issue · 5 comments
Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)
- WPF
- UWP
- iOS
- Android
- .NET Standard
- .NET Core
- .NET Framework
Component
Costura
Version of Library
5.0.0-beta0001
Version of OS(s) listed above with issue
Windows 10 version 1909
Steps to Reproduce
- Create a
sample.csprojfile with this content:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net472</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Costura.Fody" Version="5.0.0-beta0001" PrivateAssets="all" />
<PackageReference Include="Fody" Version="6.3.0" PrivateAssets="all" />
<PackageReference Include="Microsoft.Data.SqlClient" Version="2.1.1" />
</ItemGroup>
</Project>- Build the project with
MSBuild /t:restore && MSBuild -v:d
Expected Behavior
The Microsoft.Data.SqlClient dll is embedded into the executable.
Actual Behavior
The Microsoft.Data.SqlClient dll is not embedded into the executable.
We can see in the Fody logs that Microsoft.Data.SqlClient is not embedded. We can also find it in at bin\Debug\net472\Microsoft.Data.SqlClient.dll.
Note that when using Costura 4.1.0 (<PackageReference Include="Costura.Fody" Version="4.1.0" PrivateAssets="all" />) instead of version 5.0.0-beta0001, the Microsoft.Data.SqlClient dll is properly embedded.
❌ Logs when using version 5.0.0-beta0001:
FodyTarget:
Fody: Fody (version 6.3.0.0 @ file:///C:/Users/Cedric/.nuget/packages/fody/6.3.0/netclassictask/Fody.dll) Executing
Fody/Costura: Executing Weaver
Fody/Costura: Including references
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\costura.fody\5.0.0-beta0001\lib\netstandard1.0\Costura.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\costura.fody\5.0.0-beta0001\lib\netstandard1.0\Costura.pdb'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identity.client\4.21.1\lib\net461\Microsoft.Identity.Client.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.jsonwebtokens\6.8.0\lib\net461\Microsoft.IdentityModel.JsonWebTokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.logging\6.8.0\lib\net461\Microsoft.IdentityModel.Logging.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols.openidconnect\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.OpenIdConnect.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.tokens\6.8.0\lib\net461\Microsoft.IdentityModel.Tokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\system.diagnostics.diagnosticsource\4.3.0\lib\net46\System.Diagnostics.DiagnosticSource.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\system.identitymodel.tokens.jwt\6.8.0\lib\net461\System.IdentityModel.Tokens.Jwt.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\de\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\es\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\fr\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\it\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ja\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ko\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\pt-BR\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ru\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hans\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hant\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Skipping runtime references for this target framework, library doesn't target .NET Core
Fody: Finished Fody 1232ms.
✅ Logs when using version 4.1.0:
Fody: Weaver 'C:\Users\Cedric\.nuget\packages\costura.fody\4.1.0\build\..\weaver\Costura.Fody.dll'.
Fody: Initializing weaver
Fody: Loading 'C:\Users\Cedric\.nuget\packages\costura.fody\4.1.0\build\..\weaver\Costura.Fody.dll' from disk.
Fody: Weavers with 'ShouldCleanReference=true': Costura.Fody
Fody/Costura: Executing Weaver
Fody/Costura: Configuration source: c:\Projects\Experiments\costuramds\FodyWeavers.xml
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\costura.fody\4.1.0\lib\net40\Costura.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\runtimes\win\lib\net46\Microsoft.Data.SqlClient.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\runtimes\win\lib\net46\Microsoft.Data.SqlClient.pdb'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identity.client\4.21.1\lib\net461\Microsoft.Identity.Client.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.jsonwebtokens\6.8.0\lib\net461\Microsoft.IdentityModel.JsonWebTokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.logging\6.8.0\lib\net461\Microsoft.IdentityModel.Logging.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols.openidconnect\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.OpenIdConnect.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.tokens\6.8.0\lib\net461\Microsoft.IdentityModel.Tokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\system.identitymodel.tokens.jwt\6.8.0\lib\net461\System.IdentityModel.Tokens.Jwt.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\de\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\es\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\fr\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\it\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ja\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ko\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\pt-BR\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ru\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hans\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hant\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Finished 'Costura' in 947ms
And right after writing this bug report I see that Microsoft.Data.SqlClient.dll is inside the runtimes directory and this is probably why it's not embedded by Costura 5.0.0-beta0001!
So I tried to change the Costura configuration by explicitly asking to include runtime references even though it's documented to be the default with <Costura IncludeRuntimeReferences='true' />. I also tried <Costura IncludeRuntimeAssemblies='Microsoft.Data.SqlClient' /> but neither worked, the Microsoft.Data.SqlClient.dll file is still not embedded.
Is there a way to configure Costura 5.0.0-beta0001 to embed the Microsoft.Data.SqlClient dll?
This is one of the things we've been working on for Costura 5.0. We needed the embed runtime references ourselves for .NET Core / NET 5. But since .NET core can target any platform (it's not as simple as in full net fx), we needed a slightly different approach.
If you watch the assembly using IL Spy (or anything similar), you should now see a costura file with all the references, hashes, etc in a single file.
We wrote a "runtime assembly loader" for this so we can unpack and load them on the fly. The reason why we didn't embed this into Costura is because we wanted to control this ourselves when using .NET Core / .NET 5.
For the extracting of the correct runtimes:
For the app domain watcher (load on demand), see this file: https://github.com/WildGums/Orc.Extensibility/blob/develop/src/Orc.Extensibility/Watchers/AppDomainRuntimeAssemblyWatcher.cs
I understand but unfortunately I'm targeting .NET Framework and Costura completely skips runtime references on .NET Framework:
if (config.IncludeRuntimeReferences)
{
if (!ModuleDefinition.IsUsingDotNetCore())
{
WriteInfo("\tSkipping runtime references for this target framework, library doesn't target .NET Core");
}
else
{But I have found a workaround! I'm adding this in my csproj:
<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" Version="2.1.1" GeneratePathProperty="true" />
</ItemGroup>
<Target Name="TrickToEmbedMicrosoftDataSqlClientWithFody" AfterTargets="ResolveLockFileCopyLocalFiles">
<ItemGroup>
<ReferenceCopyLocalPaths Remove="@(ReferenceCopyLocalPaths)" Condition="'%(Filename)%(Extension)' == 'Microsoft.Data.SqlClient.dll'" />
<ReferenceCopyLocalPaths Include="$(PkgMicrosoft_Data_SqlClient)\lib\net46\Microsoft.Data.SqlClient.dll">
<AssetType>runtime</AssetType>
<CopyLocal>true</CopyLocal>
<DestinationSubPath>Microsoft.Data.SqlClient.dll</DestinationSubPath>
<PathInPackage>lib/net46/Microsoft.Data.SqlClient.dll</PathInPackage>
</ReferenceCopyLocalPaths>
</ItemGroup>
</Target>I am removing the resolved Microsoft.Data.SqlClient.dll whose identity is ~\.nuget\packages\microsoft.data.sqlclient\2.1.1\runtimes\win\lib\net46\Microsoft.Data.SqlClient.dll and replacing it with $(PkgMicrosoft_Data_SqlClient)\lib\net46\Microsoft.Data.SqlClient.dll, manually adding the required MSBuild item metadata (i.e. AssetType +CopyLocal etc.) which are normally computed by the ResolvePackageAssets MSBuild task.
Because runtimes\win\lib\net46\Microsoft.Data.SqlClient.dll and lib\net46\Microsoft.Data.SqlClient.dll are strictly identical files, picking one over the other is safe.
The core of this trick is that now the Microsoft.Data.SqlClient.dll is under the lib directory instead of the runtimes directory. This way, Costura doesn't consider it as a runtime reference and thus is not filtered when Costura embeds the references.
Phew! 😅
Can you please test the latest beta?
I just tested with Costura 5.0.0-beta0004 and it works. 🥳
Test project:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net472</TargetFramework>
<LangVersion>8.0</LangVersion>
<DebugType>embedded</DebugType>
<GenerateSupportedRuntime>false</GenerateSupportedRuntime>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Costura.Fody" Version="5.0.0-beta0004" PrivateAssets="all" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="5.0.0" />
</ItemGroup>
</Project>Test program:
using System;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Reflection;
using System.Text;
namespace CosturaRuntimeNetFx
{
class Program
{
static Program()
{
AppDomain.CurrentDomain.AssemblyResolve += NaiveAssemblyResolve;
}
static int Main()
{
try
{
Console.WriteLine(CodePagesEncodingProvider.Instance);
return 0;
}
catch (Exception exception)
{
Console.Error.WriteLine(exception);
return 1;
}
}
private static Assembly NaiveAssemblyResolve(object sender, ResolveEventArgs args)
{
var assembly = typeof(Program).Assembly;
var assemblyName = new AssemblyName(args.Name);
var resourceName = assembly.GetManifestResourceNames().FirstOrDefault(e => e.IndexOf(assemblyName.Name + ".dll.compressed", StringComparison.OrdinalIgnoreCase) >= 0);
Console.WriteLine($"Resolving {args.Name} with {resourceName}");
var compressedStream = assembly.GetManifestResourceStream(resourceName) ?? throw new InvalidOperationException($"The manifest resource named '{resourceName}' was not found.");
using var source = new DeflateStream(compressedStream, CompressionMode.Decompress);
var memoryStream = new MemoryStream();
source.CopyTo(memoryStream);
return Assembly.Load(memoryStream.ToArray());
}
}
}FodyWeavers.xml:
<Weavers xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="FodyWeavers.xsd">
<Costura IncludeRuntimeAssemblies="System.Text.Encoding.CodePages" />
</Weavers>The System.Text.Encoding.CodePages dll is embedded as costura.runtimes.win.lib.net461.system.text.encoding.codepages.dll.compressed and I can load it on demand with my custom NaiveAssemblyResolve method. I kept it very simple without much error handling for the sake of brevity.
Building:
MSBuild /t:restore && MSBuild
Running:
bin\Debug\net472\CosturaRuntimeNetFx.exe
Output:
Resolving System.Text.Encoding.CodePages, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a with costura.runtimes.win.lib.net461.system.text.encoding.codepages.dll.compressed
System.Text.CodePagesEncodingProvider
🕺
Yay! I say we release 5.0 as stable and see what happens :)