Can not embed Microsoft.Data.SqlClient 2.1.1 with Costura 5.0.0-beta0001
0xced opened this issue · 5 comments
Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)
- WPF
- UWP
- iOS
- Android
- .NET Standard
- .NET Core
- .NET Framework
Component
Costura
Version of Library
5.0.0-beta0001
Version of OS(s) listed above with issue
Windows 10 version 1909
Steps to Reproduce
- Create a
sample.csproj
file with this content:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net472</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Costura.Fody" Version="5.0.0-beta0001" PrivateAssets="all" />
<PackageReference Include="Fody" Version="6.3.0" PrivateAssets="all" />
<PackageReference Include="Microsoft.Data.SqlClient" Version="2.1.1" />
</ItemGroup>
</Project>
- Build the project with
MSBuild /t:restore && MSBuild -v:d
Expected Behavior
The Microsoft.Data.SqlClient
dll is embedded into the executable.
Actual Behavior
The Microsoft.Data.SqlClient
dll is not embedded into the executable.
We can see in the Fody logs that Microsoft.Data.SqlClient
is not embedded. We can also find it in at bin\Debug\net472\Microsoft.Data.SqlClient.dll
.
Note that when using Costura 4.1.0 (<PackageReference Include="Costura.Fody" Version="4.1.0" PrivateAssets="all" />
) instead of version 5.0.0-beta0001, the Microsoft.Data.SqlClient
dll is properly embedded.
❌ Logs when using version 5.0.0-beta0001:
FodyTarget:
Fody: Fody (version 6.3.0.0 @ file:///C:/Users/Cedric/.nuget/packages/fody/6.3.0/netclassictask/Fody.dll) Executing
Fody/Costura: Executing Weaver
Fody/Costura: Including references
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\costura.fody\5.0.0-beta0001\lib\netstandard1.0\Costura.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\costura.fody\5.0.0-beta0001\lib\netstandard1.0\Costura.pdb'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identity.client\4.21.1\lib\net461\Microsoft.Identity.Client.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.jsonwebtokens\6.8.0\lib\net461\Microsoft.IdentityModel.JsonWebTokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.logging\6.8.0\lib\net461\Microsoft.IdentityModel.Logging.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols.openidconnect\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.OpenIdConnect.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.tokens\6.8.0\lib\net461\Microsoft.IdentityModel.Tokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\system.diagnostics.diagnosticsource\4.3.0\lib\net46\System.Diagnostics.DiagnosticSource.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\system.identitymodel.tokens.jwt\6.8.0\lib\net461\System.IdentityModel.Tokens.Jwt.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\de\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\es\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\fr\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\it\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ja\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ko\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\pt-BR\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ru\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hans\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hant\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Skipping runtime references for this target framework, library doesn't target .NET Core
Fody: Finished Fody 1232ms.
✅ Logs when using version 4.1.0:
Fody: Weaver 'C:\Users\Cedric\.nuget\packages\costura.fody\4.1.0\build\..\weaver\Costura.Fody.dll'.
Fody: Initializing weaver
Fody: Loading 'C:\Users\Cedric\.nuget\packages\costura.fody\4.1.0\build\..\weaver\Costura.Fody.dll' from disk.
Fody: Weavers with 'ShouldCleanReference=true': Costura.Fody
Fody/Costura: Executing Weaver
Fody/Costura: Configuration source: c:\Projects\Experiments\costuramds\FodyWeavers.xml
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\costura.fody\4.1.0\lib\net40\Costura.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\runtimes\win\lib\net46\Microsoft.Data.SqlClient.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\runtimes\win\lib\net46\Microsoft.Data.SqlClient.pdb'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identity.client\4.21.1\lib\net461\Microsoft.Identity.Client.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.jsonwebtokens\6.8.0\lib\net461\Microsoft.IdentityModel.JsonWebTokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.logging\6.8.0\lib\net461\Microsoft.IdentityModel.Logging.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.protocols.openidconnect\6.8.0\lib\net461\Microsoft.IdentityModel.Protocols.OpenIdConnect.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.identitymodel.tokens\6.8.0\lib\net461\Microsoft.IdentityModel.Tokens.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\system.identitymodel.tokens.jwt\6.8.0\lib\net461\System.IdentityModel.Tokens.Jwt.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\de\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\es\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\fr\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\it\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ja\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ko\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\pt-BR\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\ru\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hans\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Embedding 'C:\Users\Cedric\.nuget\packages\microsoft.data.sqlclient\2.1.1\lib\net46\zh-Hant\Microsoft.Data.SqlClient.resources.dll'
Fody/Costura: Finished 'Costura' in 947ms
And right after writing this bug report I see that Microsoft.Data.SqlClient.dll
is inside the runtimes
directory and this is probably why it's not embedded by Costura 5.0.0-beta0001!
So I tried to change the Costura configuration by explicitly asking to include runtime references even though it's documented to be the default with <Costura IncludeRuntimeReferences='true' />
. I also tried <Costura IncludeRuntimeAssemblies='Microsoft.Data.SqlClient' />
but neither worked, the Microsoft.Data.SqlClient.dll
file is still not embedded.
Is there a way to configure Costura 5.0.0-beta0001 to embed the Microsoft.Data.SqlClient
dll?
This is one of the things we've been working on for Costura 5.0. We needed the embed runtime references ourselves for .NET Core / NET 5. But since .NET core can target any platform (it's not as simple as in full net fx), we needed a slightly different approach.
If you watch the assembly using IL Spy (or anything similar), you should now see a costura file with all the references, hashes, etc in a single file.
We wrote a "runtime assembly loader" for this so we can unpack and load them on the fly. The reason why we didn't embed this into Costura is because we wanted to control this ourselves when using .NET Core / .NET 5.
For the extracting of the correct runtimes:
For the app domain watcher (load on demand), see this file: https://github.com/WildGums/Orc.Extensibility/blob/develop/src/Orc.Extensibility/Watchers/AppDomainRuntimeAssemblyWatcher.cs
I understand but unfortunately I'm targeting .NET Framework and Costura completely skips runtime references on .NET Framework:
if (config.IncludeRuntimeReferences)
{
if (!ModuleDefinition.IsUsingDotNetCore())
{
WriteInfo("\tSkipping runtime references for this target framework, library doesn't target .NET Core");
}
else
{
But I have found a workaround! I'm adding this in my csproj:
<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" Version="2.1.1" GeneratePathProperty="true" />
</ItemGroup>
<Target Name="TrickToEmbedMicrosoftDataSqlClientWithFody" AfterTargets="ResolveLockFileCopyLocalFiles">
<ItemGroup>
<ReferenceCopyLocalPaths Remove="@(ReferenceCopyLocalPaths)" Condition="'%(Filename)%(Extension)' == 'Microsoft.Data.SqlClient.dll'" />
<ReferenceCopyLocalPaths Include="$(PkgMicrosoft_Data_SqlClient)\lib\net46\Microsoft.Data.SqlClient.dll">
<AssetType>runtime</AssetType>
<CopyLocal>true</CopyLocal>
<DestinationSubPath>Microsoft.Data.SqlClient.dll</DestinationSubPath>
<PathInPackage>lib/net46/Microsoft.Data.SqlClient.dll</PathInPackage>
</ReferenceCopyLocalPaths>
</ItemGroup>
</Target>
I am removing the resolved Microsoft.Data.SqlClient.dll
whose identity is ~\.nuget\packages\microsoft.data.sqlclient\2.1.1\runtimes\win\lib\net46\Microsoft.Data.SqlClient.dll
and replacing it with $(PkgMicrosoft_Data_SqlClient)\lib\net46\Microsoft.Data.SqlClient.dll
, manually adding the required MSBuild item metadata (i.e. AssetType
+CopyLocal
etc.) which are normally computed by the ResolvePackageAssets
MSBuild task.
Because runtimes\win\lib\net46\Microsoft.Data.SqlClient.dll
and lib\net46\Microsoft.Data.SqlClient.dll
are strictly identical files, picking one over the other is safe.
The core of this trick is that now the Microsoft.Data.SqlClient.dll
is under the lib
directory instead of the runtimes
directory. This way, Costura doesn't consider it as a runtime reference and thus is not filtered when Costura embeds the references.
Phew! 😅
Can you please test the latest beta?
I just tested with Costura 5.0.0-beta0004 and it works. 🥳
Test project:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net472</TargetFramework>
<LangVersion>8.0</LangVersion>
<DebugType>embedded</DebugType>
<GenerateSupportedRuntime>false</GenerateSupportedRuntime>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Costura.Fody" Version="5.0.0-beta0004" PrivateAssets="all" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="5.0.0" />
</ItemGroup>
</Project>
Test program:
using System;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Reflection;
using System.Text;
namespace CosturaRuntimeNetFx
{
class Program
{
static Program()
{
AppDomain.CurrentDomain.AssemblyResolve += NaiveAssemblyResolve;
}
static int Main()
{
try
{
Console.WriteLine(CodePagesEncodingProvider.Instance);
return 0;
}
catch (Exception exception)
{
Console.Error.WriteLine(exception);
return 1;
}
}
private static Assembly NaiveAssemblyResolve(object sender, ResolveEventArgs args)
{
var assembly = typeof(Program).Assembly;
var assemblyName = new AssemblyName(args.Name);
var resourceName = assembly.GetManifestResourceNames().FirstOrDefault(e => e.IndexOf(assemblyName.Name + ".dll.compressed", StringComparison.OrdinalIgnoreCase) >= 0);
Console.WriteLine($"Resolving {args.Name} with {resourceName}");
var compressedStream = assembly.GetManifestResourceStream(resourceName) ?? throw new InvalidOperationException($"The manifest resource named '{resourceName}' was not found.");
using var source = new DeflateStream(compressedStream, CompressionMode.Decompress);
var memoryStream = new MemoryStream();
source.CopyTo(memoryStream);
return Assembly.Load(memoryStream.ToArray());
}
}
}
FodyWeavers.xml:
<Weavers xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="FodyWeavers.xsd">
<Costura IncludeRuntimeAssemblies="System.Text.Encoding.CodePages" />
</Weavers>
The System.Text.Encoding.CodePages
dll is embedded as costura.runtimes.win.lib.net461.system.text.encoding.codepages.dll.compressed
and I can load it on demand with my custom NaiveAssemblyResolve
method. I kept it very simple without much error handling for the sake of brevity.
Building:
MSBuild /t:restore && MSBuild
Running:
bin\Debug\net472\CosturaRuntimeNetFx.exe
Output:
Resolving System.Text.Encoding.CodePages, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a with costura.runtimes.win.lib.net461.system.text.encoding.codepages.dll.compressed
System.Text.CodePagesEncodingProvider
🕺
Yay! I say we release 5.0 as stable and see what happens :)