jstedfast/MimeKit

Getting exception sending emails with MimeKit, after upgrading from 3.4.2 to 3.4.3

ajtruckle opened this issue · 37 comments

ajtruckle commented

Can anyone sched any light on this exception that was triggered on a users computer when my DLL was used to try and send an email?

Exception:

<LogEntry Date="2022-12-23 05:57:59" Severity="Exception" Source="MSAToolsGMailLibrary.MSAToolsGMailLibraryClass.SendEmail" ThreadId="1">
    <Exception Type="System.IO.FileNotFoundException" Source="System.Span`1..ctor">
    <Message>Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.</Message>
         <StackTrace>   at System.Span`1..ctor(T[] array)
                        at MimeKit.Utils.ValueStringBuilder..ctor(Int32 initialCapacity)
                        at MimeKit.Utils.Rfc2047.Encode(FormatOptions options, Encoding charset, String text, Boolean phrase)
                        at MimeKit.Utils.Rfc2047.EncodeText(FormatOptions options, Encoding charset, String text)
                        at MimeKit.Header.EncodeUnstructuredHeader(ParserOptions options, FormatOptions format, Encoding encoding, String field, String value)
                        at MimeKit.Header.EncodeAddressHeader(ParserOptions options, FormatOptions format, Encoding encoding, String field, String value)
                        at MimeKit.Header.FormatRawValue(FormatOptions format, Encoding encoding, String value)
                        at MimeKit.Header.SetValue(FormatOptions format, Encoding encoding, String value)
                        at MimeKit.Header.SetValue(Encoding encoding, String value)
                        at MimeKit.Header..ctor(Encoding encoding, HeaderId id, String value)
                        at MimeKit.HeaderList.set_Item(HeaderId id, String value)
                        at MimeKit.MimeMessage..ctor()
                        at MSAToolsGMailLibrary.MSAToolsGMailLibraryClass.SendEmail(String From, String Subject, String Attachment) 
        </StackTrace>
    </Exception>
</LogEntry>
ajtruckle commented

I just noticed that this issue is also affecting me too now:

Exception

Not sure how to address this.

ajtruckle commented

I see this realest discussion on the exception:

https://stackoverflow.com/questions/62764744/could-not-load-file-or-assembly-system-runtime-compilerservices-unsafe

I have Unsafe version 6.0.0 installed. But it was all handled by the NuGet Package Manager.

Please advise. Thank you.

ajtruckle commented

I tried adding this to the config file and it seems to make no difference:

			<dependentAssembly>
				<assemblyIdentity name="System.Runtime.CompilerServices.Unsafe" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="6.0.0.0"/>
			</dependentAssembly>
ajtruckle commented

Update:

If I downgrade MimeKit from 3.4.3 back to 3.4.2 then my DLL functions fine.

Await update.

jstedfast commented

MimeKit v3.4.3 was my best attempt at fixing this and it solved it for the people that hit this problem using 3.4.2 and older versions.

But you are saying the opposite?

I'm stumped. I don't have this problem and have never been able to reproduce it.

It has to be a conflict with another library you are depending on.

ajtruckle commented

Possibly ... I have a few projects that all use NuGet packages. To my knowledge, only two of the projects specifically mention this "Unsafe" NuGet package, and both are on 6.0.0. So I am slightly confused myself. The others that have NuGet packages do not use "Unsafe" from what I can tell.

I am not well knowledges with the mechanics of NuGet packages. If you want me to look for certain data about my projects setups and / or try anything specific then let me know.

jstedfast commented

This is a duplicate of jstedfast/MailKit#1462

The solution in that is to use the following App.config:

<dependentAssembly>
    <assemblyIdentity name="System.Runtime.CompilerServices.Unsafe" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
    <bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="4.0.4.1" />
</dependentAssembly>
jstedfast commented

Note: The NuGet package version 6.0.0 of System.Runtime.CompilerServices.Unsafe does not equate to the Assembly Version which is likely 4.0.4.1 which is why your original App.config does not work. You need to map the Assembly Version and not the NuGet Version.

ajtruckle commented
ajtruckle commented
ajtruckle commented

I don't get it. So if 4.5.3 IS 4.0.4.1 ... and I have that in the app.config, why the exception?

I did check both output folders and they were the same time stamps.

jstedfast commented

Welcome to DLL Hell? I don't know.

ajtruckle commented

I think I might know what is happening. I will try later today and come back with feedback.

ajtruckle commented

I am giving up on this for now.

Sub project A has a reference to System.Memory. And it has a binding redirect to use 4.0.1.2 (which is the latest). There is no entry for the Unsafe dependency. It has the latest 6.0.0 downloaded. Admittedly the System.Memory says the dependancy is >= 4.5.3 so this is probably why it works. And, the date of the DLL is 19/02/2020.

Now, sub project B is the one with MimeKit. One issue here is that with 3.4.3 you have:

System.Memory >= 4.5.5
Unsafe >= 6.0.0

I don't see why you have done this. In 3.4.2 there was not even a dependency on Unsafe. In System.Memory needs >= 4.5.3 then MimeKit should be >= 4.5.3. Why is it forcing this conflict? I have to stick with 3.4.2 because it is not breaking the system and I need no binding redirects. Life is simple.

Can't you simply change it to >= 4.5.3? To be consistent?

jstedfast commented

If your suggestion would fix the issue, then what explains this? jstedfast/MailKit#1462

In other words, other people were getting the exact same problem you are seeing long before I made the explicit dependency on Unsafe >= 6.0.0. The explicit dependency on 6.0.0 was, in fact, an attempt at fixing this issue or at the very least, making it easier to resolve because if MimeKit depends on the latest, then the compiler should bring in 6.0.0 and all of the other libraries should Just Work(tm) if they depend on >= 4.0.4.1 or whatever the version is.

I made the dependency on >= 6.0.0 explicit because other NuGet packages that other people are using have a dependency on Unsafe >= 6.0.0.

If System.Memory was the only NuGet package that depended on Unsafe, then I would not make an explicit dependency and things would likely work as long as no one explicitly added a reference to Unsafe that did not match the exact version used by System.Memory.

Unfortunately, that's not something I can depend on happening.

jstedfast commented

BTW, it might help if I knew which TargetFramework you are targeting. Maybe there's an issue with whatever framework you are using but not the others?

ajtruckle commented

I am targeting Framework 4.6.2 and don't want to change that.

ajtruckle commented

I have only one other package that specifically says dependency of >= 4.5.3 and that is System.Memory. And all of my sub tools are consistent with using 4.6.2 Framework.

I still don;t se why this is forced. There is no specific reason for >= 6.0.0. After all, the package I use (which is last updated in May this year) is not forcing it. We are not all using 4.8 frameworks for various reasons.

In the linked discussion he stated:

I have System.Runtime.CompilerServices.Unsafe, Version=6.0.0.0 installed on my system, yet MailKit is referencing System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1 which is not installed.

Yet, if your dependency is 4.0.4.1 OR HIGHER then it should not matter if he had a newer Unsafe should it? That's the real problem to have got fixed somehow IMHO.

I have to find a common ground too against all the nuget packages and until system.memory also uses 6.0.0 then I can't upgrade. And framework 4.6.2 may be a big factor.

jstedfast commented

Yet, if your dependency is 4.0.4.1 OR HIGHER then it should not matter if he had a newer Unsafe should it? That's the real problem to have got fixed somehow IMHO.

It shouldn't, but clearly it does or you wouldn't have the problem you are hitting, either, would you?

That's the problem.

If MimeKit depends on 6.0.0 and System.Memory depends on >= 4.5.3, then how are you hitting this issue?

Likewise, if MimeKit depends on >= 4.5.3 and System.Memory depends on >= 4.5.3 and this other guy's project depends on Unsafe >= 6.0.0, then MimeKit/System.Memory should both Just Work(tm), but they don't.

WHY???

The issue is that unless everything depends on the same version of Unsafe, then at least 1 of the assemblies will break.

ajtruckle commented

Yeah, I agree. Well, I will just stay with 3.4.2 for now as it works. I'll keep an eye on when System.Memory upgrades and I will deal with it then.

jstedfast commented

There seems to be a blog explaining things here: https://nickcraver.com/blog/2020/02/11/binding-redirects/

Marc Gravell notes that this is an MSBuild issue in one of the comments in a StackExchange.Redis bug report with the same issue you are hitting in MimeKit: StackExchange/StackExchange.Redis#1128 (comment)

ajtruckle commented

Yeah. Thanks for that. So:

  • Am I supposed to add this "binding redirect to 4.0.4.1" to all of the 3 projects that refer to the Unsafe DLL?

I do install the Dlls (like Unsafe) along with my DLL in the respective folders on their PC. On mine I get it to copy them to my development folders. I am using Admin when using VS.

jstedfast commented

I think if you add an explicit dependency on Unsafe to your project, then VS will handle creating the app.config file for you.

I think you only need it for the app project (and not any of the library projects you may have).

ajtruckle commented
jstedfast commented

The project that contains your main executable as far as I know.

ajtruckle commented

Gotcha, but as mentioned, my master application is a C++ MFC project. It does not use NuGet packages nor have a app.config file.

I spent some time re-working my sub projects so that they all end up in their own sub-folders of the main application. This way, their respective DLL files are distinct.

I have a total of four sub projects:

  • Google Sync

VB.Net It does not use the Unsafe DLL and is a standalone executable (console app). So it is not affected.

  • Outlook Sync

C#.Net It indirectly uses the Unsafe DLL (NuGet 6.0.0). It does so indirectly through the System.Memory dependency. It too is a standalone console app. The app.config looks like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <configSections>
    </configSections>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.2"/>
    </startup>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Net.Http" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.2.0.0" newVersion="4.1.1.3"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Graph.Core" publicKeyToken="31bf3856ad364e35" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-1.20.0.0" newVersion="1.20.0.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-13.0.0.0" newVersion="13.0.0.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.ValueTuple" publicKeyToken="cc7b13ffcd2ddd51" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.0.3.0" newVersion="4.0.3.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Numerics.Vectors" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.1.4.0" newVersion="4.1.4.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Buffers" publicKeyToken="cc7b13ffcd2ddd51" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.0.3.0" newVersion="4.0.3.0"/>
      </dependentAssembly>
	</assemblyBinding>
  </runtime>
</configuration>

Date of unsafe DLL is 23/10/2021.

  • Tools Dll

C#.Net COM Has NuGet Unsafe 6.0.0 installed. Only needed because of System.Memory.

app.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<runtime>
		<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
			<dependentAssembly>
				<assemblyIdentity name="System.ValueTuple" publicKeyToken="cc7b13ffcd2ddd51" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-4.0.3.0" newVersion="4.0.3.0" />
			</dependentAssembly>
			<dependentAssembly>
				<assemblyIdentity name="System.Memory" publicKeyToken="cc7b13ffcd2ddd51" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-4.0.1.2" newVersion="4.0.1.2" />
			</dependentAssembly>
		</assemblyBinding>
	</runtime>
</configuration>

Date of unsafe DLL is 23/10/2021.

  • Gmail Tools Dll

C#.Net COM Has NuGet Unsafe 4.5.3 installed. Only needed because of System.Memory. Or so I thought. But if I upgrade it to 6.0.0 and run my software, the specific feature still fails, even though MimeKit was still 3.4.2. So it just doesn't like have 6.0.0.

But, the other projects also have System.Memory and work fine with Unsafe Nuget 6.0.0. But not this one.

So, I have come to the conclusion that the main culprit here might be:

  • Google.Apis.Gmail.v1

Gmail

I think it is this tool that forces it. My PackageReference for this project:

    <PackageReference Include="BouncyCastle">
      <Version>1.8.9</Version>
    </PackageReference>
    <PackageReference Include="Google.Apis.Gmail.v1">
      <Version>1.58.0.2650</Version>
    </PackageReference>
    <PackageReference Include="MimeKit">
      <Version>3.4.2</Version>
    </PackageReference>
    <PackageReference Include="Portable.BouncyCastle">
      <Version>1.9.0</Version>
    </PackageReference>
    <PackageReference Include="System.Runtime.CompilerServices.Unsafe">
      <Version>4.5.3</Version>
    </PackageReference>

I am curious. If you try adding Google.Apis.Gmail.v1 and are targeting framework 4.6.2, do you start to get issues?

And as previously mentioned, using a redirect on my app is not sufficient. I reckon it is somthing else under the hood - maybe deep causing the issue.

So I have to stay with 4.5.3.

ajtruckle commented

Nuget 4.5.3 will install 19/02/2020 version.
Nuget 6.0.0 will install 23/10/2021 version.

So there is no 4.5.3 assembly to redirect to?

ajtruckle commented

Hang on, what about:

  1. Upgrade Mimekit.
  2. Downgrade Unsafe back to 4.5.3
  3. Add binding redirect 0000 to 6000 to use 4041

I reckon that may work as I would still have 453 dll for unsafe.

ajtruckle commented

BuildError

If I upgrade MimeKit and Downgrade Unsafe, it won't compile as you can see. Even if I add a binding redirect.

ajtruckle commented

Hi! I have reached out to Google team.

googleapis/google-api-dotnet-client#2297

The thing is that the Gmail Nuget package supports framework 4.5 or higher. All others seem to be 4.6.2. Makes sense since 4.6.1 is deprecated. So maybe that is why it is forcing it.

But in my initial tests with a c# console app using our code it was fine. Odd! 😬 So next is to actually try and do this as c# com dll and call from c++.

ajtruckle commented

@jstedfast
Hi, in a linked discussed they suggest:

It sounds like you may be in a scenario where assembly binding redirects are infeasible - in which case I suspect you're stuck, basically. The modern .NET ecosystem relies on the ability to use dependencies which depend on different versions of the same package, so long as the version required version is still compatible with the most recent required version. The somewhat-legacy .NET Framework usually handles this with assembly binding redirects; .NET Core and .NET have a slightly different mechanism. But if you're in a situation where those facilities aren't available, you're effectively requiring a single set of transitive dependencies where nothing depends on a different version from the rest of the set. If you can find a set of mutually-compatible versions, that's great - but if you can't, there's nothing library authors can really do to help you.

I do believe this is the case we are in.

jstedfast commented

Ouch, yea, sounds like it.

joaocib commented

I was facing a back and forth war between System.Rutime.CompilerServices.Unsafe and System.Memory. I have tried many possibilities you may think of. I did manage to solve right now (months later) by simple updating System.Memory to 4.5.5. I'm targeting .net462 and using Costura, just to mention.

ajtruckle commented

@joaocib I never got to the bottom of it. In the end I re-wrote my tool with a brand new .Net8 Console app and everything worked OK.

jstedfast commented

FWIW, a while back I added MimeKit.dll.config files to the nuget packages which provide the assembly redirects which are supposedly the work-around to fix this, but obviously if you are still hitting this, then I don't know what the solution could be.