Certain System.* not being embedded?
EricZimmerman opened this issue Β· 29 comments
Hello!
I am an active supporter of Fody. i am using costura v5.0.2
I noticed recently that certain things like the following do not seem to be getting rolled into the exe like they used to.
System.Text.Encoding.CodePages.dll
this leads end users to run into issues when they use the software. I do see plenty of other System.* stuff in the obj folder tho
if I tweak my xml config with this
<Costura>
<IncludeAssemblies>
System.Text.*
AlphaFS
CsvHelper
Exceptionless
FluentCommandLineParser
GuidMapping
Microsoft.Win32.Primitives
NFluent
NLog
RawCopy
Registry
ServiceStack.Text
ShellBagsParser
System.Buffers
System.Numerics.Vectors
System.Runtime.CompilerServices.Unsafe
</IncludeAssemblies>
</Costura>
I CAN see this file here and compressed, as expected:
obj\Release\Costura
but if I have a default xml config, that file does not show up in obj\Release\Costura
I also always seem to get this warning:
Fody: The package reference for Costura.Fody is missing the 'compile' part in the IncludeAssets setting; it's recommended to completely remove IncludeAssets
which i remove but it does not seem to help.
I saw #652 and thought that would do it. i assume there is not a newer version out there later than 5.0.2?
any idea why this is happening and how to fix it?
5.0.2 is the latest version available.
- What version of .NET are you targeting?
- How are you using Fody & Costura in your csproj file?
- .net 4.6.2 to maybe 4.7 here and there, but the projects i have seen this recently are 4,6 based. Not doing anything .net core with fody\custura
- its as simple as it gets, and whatever happens when nuget adds them to the project. one looks like this (other stuff removed). I did remove the IncludeAssets tag per the warning. I always had an empty FodyWeavers.xml file as well, but only recently started experimenting with having to add the names via IncludeAssemblies. usually its just
<ItemGroup>
<PackageReference Include="Costura.Fody">
<Version>5.0.2</Version>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Fody">
<Version>6.4.0</Version>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
btw i am using Version 16.9.1 of VS Pro
here is another example from a different project, showing the defaults after adding from nuget:
<ItemGroup>
<PackageReference Include="Costura.Fody">
<Version>5.0.2</Version>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="CsvHelper">
<Version>26.0.1</Version>
</PackageReference>
<PackageReference Include="Exceptionless">
<Version>4.6.2</Version>
</PackageReference>
<PackageReference Include="FluentCommandLineParser">
<Version>1.4.3</Version>
</PackageReference>
<PackageReference Include="Fody">
<Version>6.4.0</Version>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="MaxMind.GeoIP2">
<Version>4.0.1</Version>
</PackageReference>
<PackageReference Include="Microsoft.CSharp">
<Version>4.7.0</Version>
</PackageReference>
<PackageReference Include="NLog">
<Version>4.7.8</Version>
</PackageReference>
</ItemGroup>
this too results in files remaining in my debug folder on compile:
System.IO.Compression.dll
System.Net.Http.dll
System.Runtime.InteropServices.RuntimeInformation.dll
System.Security.Cryptography.Algorithms.dll
System.Security.Cryptography.Encoding.dll
System.Security.Cryptography.X509Certificates.dll
If you view your assembly with IL Spy, you should see the costura embedded info under resources. Can you please post that as well?
Can you check costura.metadata contents and post it here?
And it would be best if there is some sort of repro so I can try to reproduce. How is System.Security.Cryptography.X509Certificates
getting there without being referenced (so probably there are some implicit dependencies that Costura is not picking up correctly).
And, one more thing: you can enable debug logging (verbose) for ms build and see what Costura logs to the output window, it might give some useful hints.
use this repo. same one i am working with above. this is not the same project i mentioned initially about codepages, but i am guessing its all related. i suppose adding a ref to
System.Text.Encoding.CodePage
would let you see that happen in this project as well.
https://github.com/EricZimmerman/iisGeolocate
as to where the 509 thing is coming from, not sure, but i know i have nuget with the codepages reference in it and that dll gets dropped too.
costura.metadata
costura.costura.dll.compressed|5.0.2.0|Costura, Version=5.0.2.0, Culture=neutral, PublicKeyToken=null|Costura.dll|DC4B6E415DF4BAFDA5C2A6228D465D64383A8DAF
costura.costura.pdb.compressed|||Costura.pdb|51DBB96092007F8A2F4551B197DA996E4B48C217
costura.csvhelper.dll.compressed|26.0.0.0|CsvHelper, Version=26.0.0.0, Culture=neutral, PublicKeyToken=8c4959082be5c823|CsvHelper.dll|87A29163204E9FEFF52B6D6E6815ACD749F35A62
costura.exceptionless.dll.compressed|4.0.0.0|Exceptionless, Version=4.0.0.0, Culture=neutral, PublicKeyToken=fc181f0a46f65747|Exceptionless.dll|FC800C27F4D5F9DD0386389327B7F4FF206EEA09
costura.exceptionless.pdb.compressed|||Exceptionless.pdb|617CBFA2CC444A95ED3DF98FCE5A36E4BFC9054F
costura.fluentcommandlineparser.dll.compressed|1.4.3.0|FluentCommandLineParser, Version=1.4.3.0, Culture=neutral, PublicKeyToken=null|FluentCommandLineParser.dll|553EC4ECC90676C7BB1DE9F75A6B1226F39677AA
costura.fluentcommandlineparser.pdb.compressed|||FluentCommandLineParser.pdb|5753997682C6B57B8F8A946382220492D45665BD
costura.maxmind.db.dll.compressed|2.0.0.0|MaxMind.Db, Version=2.0.0.0, Culture=neutral, PublicKeyToken=66afa4cc5ae853ac|MaxMind.Db.dll|96DB62E4BC50F390EAD70AAAF8A9942B2EA857DF
costura.maxmind.geoip2.dll.compressed|4.0.0.0|MaxMind.GeoIP2, Version=4.0.0.0, Culture=neutral, PublicKeyToken=66afa4cc5ae853ac|MaxMind.GeoIP2.dll|46CC26D014261DA66DE45CECF9BEAB44ABCE0566
costura.microsoft.bcl.asyncinterfaces.dll.compressed|5.0.0.0|Microsoft.Bcl.AsyncInterfaces, Version=5.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|Microsoft.Bcl.AsyncInterfaces.dll|5DBF1534FFBFF016CC45559EB5EFF3DC4252A522
costura.microsoft.win32.primitives.dll.compressed|4.0.2.0|Microsoft.Win32.Primitives, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|Microsoft.Win32.Primitives.dll|AFA308C0166D8CBDD1D5E6FDAA8C9B87EB9184E7
costura.nlog.dll.compressed|4.0.0.0|NLog, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5120e14c03d0593c|NLog.dll|13E601EC1253B135BC4E5B64442AB443DB172E0E
costura.system.appcontext.dll.compressed|4.0.0.0|System.AppContext, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.AppContext.dll|C9A0C7200AD55960A1E1824B04718CBF6CA84581
costura.system.buffers.dll.compressed|4.0.3.0|System.Buffers, Version=4.0.3.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.Buffers.dll|2F410A0396BC148ED533AD49B6415FB58DD4D641
costura.system.collections.immutable.dll.compressed|1.2.5.0|System.Collections.Immutable, Version=1.2.5.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Collections.Immutable.dll|88DF86EE0B137992AF15A35825804274FA252E30
costura.system.console.dll.compressed|4.0.1.0|System.Console, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Console.dll|B5102629939EBDD4BCD0F407A92828DEE5F2B565
costura.system.diagnostics.diagnosticsource.dll.compressed|4.0.1.0|System.Diagnostics.DiagnosticSource, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.Diagnostics.DiagnosticSource.dll|85DC92EDD4B0049ED9049E075C4DEF8A3D64E43B
costura.system.diagnostics.tracing.dll.compressed|4.1.1.0|System.Diagnostics.Tracing, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Diagnostics.Tracing.dll|5340B1FC77E793D1FDDE49B14B7B426D0E448870
costura.system.globalization.calendars.dll.compressed|4.0.2.0|System.Globalization.Calendars, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Globalization.Calendars.dll|30AE8C8E57BCABE27BA675FA84D4DA6AF3C10EAD
costura.system.io.dll.compressed|4.1.1.0|System.IO, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.IO.dll|BA5F2CAA404AE9A283A841C08B024BAF8817BA01
costura.system.io.compression.zipfile.dll.compressed|4.0.2.0|System.IO.Compression.ZipFile, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b77a5c561934e089|System.IO.Compression.ZipFile.dll|99D6C187DE8510206A93D2EED9C65E65E0C86E72
costura.system.io.filesystem.dll.compressed|4.0.2.0|System.IO.FileSystem, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.IO.FileSystem.dll|B46AD93384323ED4AA88E8AF3CDEE208A88F1F6D
costura.system.io.filesystem.primitives.dll.compressed|4.0.2.0|System.IO.FileSystem.Primitives, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.IO.FileSystem.Primitives.dll|8374975CA07A300021EF0ACAA4F908D5BFB102FB
costura.system.memory.dll.compressed|4.0.1.1|System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.Memory.dll|19BF79600B716523B5317B9A7B68760AE5D55741
costura.system.net.sockets.dll.compressed|4.1.1.0|System.Net.Sockets, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Net.Sockets.dll|00278D53E59026CD8C22889773466FEF27D52BBF
costura.system.numerics.vectors.dll.compressed|4.1.4.0|System.Numerics.Vectors, Version=4.1.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Numerics.Vectors.dll|3D216458740AD5CB05BC5F7C3491CDE44A1E5DF0
costura.system.reflection.dll.compressed|4.1.1.0|System.Reflection, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Reflection.dll|B830E365EFC12167714E591DC7C1DD0A623FCA48
costura.system.reflection.metadata.dll.compressed|1.4.5.0|System.Reflection.Metadata, Version=1.4.5.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Reflection.Metadata.dll|6F0F1BA6DF9379DDC9DE1D586286F621AF69C298
costura.system.runtime.dll.compressed|4.1.1.0|System.Runtime, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Runtime.dll|7E549944512DE844B49AAD4B9D4A9547C06EDE37
costura.system.runtime.compilerservices.unsafe.dll.compressed|5.0.0.0|System.Runtime.CompilerServices.Unsafe, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Runtime.CompilerServices.Unsafe.dll|BAB44612721C3DC91AC3D9DFCA7C961A3A511508
costura.system.runtime.extensions.dll.compressed|4.1.1.0|System.Runtime.Extensions, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Runtime.Extensions.dll|9C6772A60E6464208A491A2CB55D62C8F3C67D00
costura.system.runtime.interopservices.dll.compressed|4.1.0.0|System.Runtime.InteropServices, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Runtime.InteropServices.dll|452039516E53E284D9DB5112CD1D5D50A954C1E2
costura.system.security.cryptography.primitives.dll.compressed|4.0.1.0|System.Security.Cryptography.Primitives, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Security.Cryptography.Primitives.dll|62A622557CC0D6FCED9C1A14BE28DBC39E9BD6FC
costura.system.text.encodings.web.dll.compressed|5.0.0.0|System.Text.Encodings.Web, Version=5.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.Text.Encodings.Web.dll|23A0B63A62A6F0907DAF759088A2A489990FCF2F
costura.system.text.json.dll.compressed|5.0.0.0|System.Text.Json, Version=5.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.Text.Json.dll|DF505C2FF55722D3F552D4A7A72C9FCADFB5B60C
costura.system.threading.tasks.extensions.dll.compressed|4.2.0.1|System.Threading.Tasks.Extensions, Version=4.2.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.Threading.Tasks.Extensions.dll|2242627282F9E07E37B274EA36FAC2D3CD9C9110
costura.system.valuetuple.dll.compressed|4.0.3.0|System.ValueTuple, Version=4.0.3.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51|System.ValueTuple.dll|81CD1ACB53D493C54C8D56F379D790A901A355AC
costura.system.xml.readerwriter.dll.compressed|4.1.0.0|System.Xml.ReaderWriter, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a|System.Xml.ReaderWriter.dll|765C9E4D36B4F2986514C1AB24B5E5E56AFFF448
diagnostic build
Costura 5 has changed the way it embeds packages which have runtime
assemblies. It is still possible to handle those System.* assemblies with Costura 5 but it involves a lot more work than with Costura 4. I have written a full explanation with sample code in #651 (comment). The pain point is that you have to write your own AssemblyResolve callback to make it work now. π
This is very unfortunate, especially since the dll files which are inside the runtimes
directories are byte for byte equivalent to the libraries inside the lib
directories. (At least for all those that I have tested.)
comp %USERPROFILE%\.nuget\packages\system.text.encoding.codepages\5.0.0\runtimes\win\lib\net461\System.Text.Encoding.CodePages.dll %USERPROFILE%.nuget\packages\system.text.encoding.codepages\5.0.0\lib\net461\System.Text.Encoding.CodePages.dll
Files compare OK
comp %USERPROFILE%\.nuget\packages\system.security.cryptography.x509certificates\4.3.0\runtimes\win\lib\net461\System.Security.Cryptography.X509Certificates.dll %USERPROFILE%.nuget\packages\system.security.cryptography.x509certificates\4.3.0\lib\net461\System.Security.Cryptography.X509Certificates.dll
Files compare OK
I'm using Costura on .NET Framework exclusively (I use single file executables on .NET Core) and I'd prefer to have Costura 4 behavior when it comes to embedding runtime references.
@GeertvanHorrik, would you accept a pull request that restores the Costura 4 behavior on .NET Framework assemblies?
For full .NET => yes, for .NET Core => no
this would be most welcome. i too have a lot of "legacy" .net stuff and would benefit from this.
I'm working on it! Gotta finish my GitHub sponsoring page. π
The pull request is ready for review: #679. It basically restores the Costura 4 behavior for embedding runtime assemblies on .NET Framework while preserving the new behavior for .NET Core. All this is configurable through the new UseRuntimeReferencePaths
configuration setting.
@0xced did a really good job here, well done! Will release a prerelease now. Please let us know if it's all good so we can release 5.1.
ill give it a try. based on the change, it sounds like plain old
<Costura/>
will do the trick, like in days of old
Correct. The version is here: https://www.nuget.org/packages/Costura.Fody/5.1.0-alpha0409
I can see that there's still a pdb and a config file containing only supported runtime and binding redirects. If you want to get rid of the pdb as a separate file but still want the stack trace with source code lines information you can use the following setting in your csproj file. I like to do this to ensure that the pdb matches the exe so that I'm 100% sure the line numbers in stack traces are correct.
<PropertyGroup>
<DebugType>embedded</DebugType>
</PropertyGroup>
And if you want to get rid of the config file, delete it and add the following settings in your csproj file so that it's not automatically recreated:
<PropertyGroup>
<AutoGenerateBindingRedirects>false</AutoGenerateBindingRedirects>
<GenerateSupportedRuntime>false</GenerateSupportedRuntime>
</PropertyGroup>
Binding redirects are useless when the dll is embedded with Costura and I think it's not a high risk to remove the supported runtime setting which target .NET Framework 4.6.2. I'm not really sure what this does, I think it might prompt the user to install the required version of .NET framework if it's not installed.
that will be awesome. would love to have actual line numbers in release.
after reloading the project, the warning is back
Fody: The package reference for Costura.Fody is missing the 'compile' part in the IncludeAssets setting; it's recommended to completely remove IncludeAssets
any reason why this cant be fixed somehow? what exactly does this even mean? does not seem to harm anything if its there, or not.
that will be awesome. would love to have actual line numbers in release.
You will, even if you have to use CreateTemporaryAssemblies='true'
. π Have a look at #633 if you like gory technical details. π
the warning is back
Hereβs how to fix it (remove the IncludeAssets
element):
<ItemGroup>
<PackageReference Include="Costura.Fody">
<Version>5.1.0-alpha0409</Version>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
yep, thats what i have been doing, removing it.
Just published 5.1 stable.
that will be awesome. would love to have actual line numbers in release.
Actually, you need .NET Framework 4.7.2 to get line numbers with portable/embedded pdb, see Stack traces are missing source information for frames with debug information in the Portable PDB format when running on .NET Framework 4.7.1 and Retargeting Changes for Migration from .NET Framework 4.7.1 to 4.7.2
Thanks for releasing 5.1.0 stable, Geert. Could you please also create releases on Costura's GitHub releases page, ideally writing some release notes in them?
The deployment pipeline should automatically create the releases (drafts) with the tickets if we were created the milestones and added the tickets there. Now it's a manual process, will see what we can do about htis in the future.
Created milestones for 5.2.0 and assigned the tickets. It should auto create the release notes next time.