Scripts using PNP.PowerShell do not work in VSCode
Tom-CK opened this issue · 65 comments
Prerequisites
- I have written a descriptive issue title.
- I have searched all issues to ensure it has not already been reported.
- I have read the troubleshooting guide.
- I have verified that I am using the latest version of Visual Studio Code and the PowerShell extension.
Summary
I've installed PNP.PowerShell-Module Version 1.7.0.
When working within a normal PowerShell, all CMDLets work fine - for example running "Get-PBPListItem" returns the expected items.
When I work in VSCode with installed PowerShell Extension (v 2021.8.0), ALL PNP-PowerShell CMDLets throw an exception (TargetInvocationException).
This used to work perfect earlier ....
PowerShell Version
Name Value
---- -----
PSVersion 5.1.19041.1023
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.19041.1023
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1Visual Studio Code Version
1.59.0
379476f0e13988d90fab105c5c19e7abc8b1dea8
x64Extension Version
ms-vscode.powershell@2021.5.1
ms-vscode.powershell@2021.8.0
ms-vscode.powershell-preview@2021.8.0Steps to Reproduce
- Install PNP.PowerShell module
- run a CMDLet like Get-PNListItem in a regular powershell window -> working !
- run a CMDLet like Get-PNListItem in a vscode powershell window -> TargetInvocationException
Visuals
Logs
No response
Can you provide the output of $error[0] | fl * -force after the error occurs?
$error[0] | fl * -force
writeErrorStream : True
PSMessageDetails :
Exception : System.Reflection.TargetInvocationException: Ein Aufrufziel hat einen Ausnahmefehler verursacht. ---> System.MissingMethodException:
Methode nicht gefunden: "Int32 System.Text.Encodings.Web.TextEncoder.FindFirstCharacterToEncodeUtf8(System.ReadOnlySpan`1<Byte>)".
bei System.Text.Json.JsonWriterHelper.NeedsEscaping(ReadOnlySpan`1 value, JavaScriptEncoder encoder)
bei System.Text.Json.JsonEncodedText.EncodeHelper(ReadOnlySpan`1 utf8Value, JavaScriptEncoder encoder)
bei System.Text.Json.JsonPropertyInfo.DeterminePropertyName()
bei System.Text.Json.JsonPropertyInfo.GetPolicies()
bei System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, Type
implementedPropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonConverter converter, JsonSerializerOptions options)
bei System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions
options)
bei System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
bei System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
bei System.Text.Json.JsonSerializer.ReadCore(Type returnType, JsonSerializerOptions options, Utf8JsonReader& reader)
bei System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
bei System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
bei PnP.Framework.Utilities.TokenHelper.GetMetadataDocument(String realm)
bei PnP.Framework.Utilities.TokenHelper.GetStsUrl(String realm)
bei PnP.Framework.Utilities.TokenHelper.GetAppOnlyAccessToken(String targetPrincipalName, String targetHost, String targetRealm)
bei PnP.Framework.Utilities.ACSTokenGenerator.GetToken(Uri siteUrl)
bei PnP.Framework.AuthenticationManager.<>c__DisplayClass75_0.<GetAccessTokenContext>b__0(Object sender, WebRequestEventArgs args)
bei Microsoft.SharePoint.Client.ClientRuntimeContext.OnExecutingWebRequest(WebRequestEventArgs args)
--- Ende der internen Ausnahmestapelüberwachung ---
bei System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
bei System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
bei System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters,
CultureInfo culture)
bei Microsoft.SharePoint.Client.ClientContextExtensions.<>c__DisplayClass10_0.<Clone>b__0(Object oSender, WebRequestEventArgs
webRequestEventArgs)
bei System.EventHandler`1.Invoke(Object sender, TEventArgs e)
bei Microsoft.SharePoint.Client.ClientRuntimeContext.OnExecutingWebRequest(WebRequestEventArgs args)
bei Microsoft.SharePoint.Client.ClientContext.GetWebRequestExecutor()
bei Microsoft.SharePoint.Client.ClientContext.<GetFormDigestInfoPrivateAsync>d__37.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei Microsoft.SharePoint.Client.ClientContext.<EnsureFormDigestAsync>d__36.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei Microsoft.SharePoint.Client.ClientContext.<ExecuteQueryAsync>d__28.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei Microsoft.SharePoint.Client.ClientContextExtensions.<ExecuteQueryImplementation>d__6.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei Microsoft.SharePoint.Client.ClientContextExtensions.ExecuteQueryRetry(ClientRuntimeContext clientContext, Int32 retryCount,
String userAgent)
bei PnP.PowerShell.Commands.PnPWebCmdlet.GetWeb()
bei PnP.PowerShell.Commands.Lists.GetListItem.ExecuteCmdlet()
bei PnP.PowerShell.Commands.PnPSharePointCmdlet.ProcessRecord()
TargetObject :
CategoryInfo : WriteError: (:) [Get-PnPListItem], TargetInvocationException
FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Lists.GetListItem
ErrorDetails : Ein Aufrufziel hat einen Ausnahmefehler verursacht.
InvocationInfo : System.Management.Automation.InvocationInfo
ScriptStackTrace : bei <ScriptBlock>, <Keine Datei>: Zeile 1
PipelineIterationInfo : {0, 1}
Maybe a typo in a configuration ?
I did not change anything in config.
Can you confirm that $PSVersion from the integrated console session itself? I'm wondering if maybe you're in PowerShell 7 there...(I mean, it should still work, but it would help narrow things down).
@rjmholt Look at the accepted answer in https://docs.microsoft.com/en-us/answers/questions/302548/jsonserializer-throws-exception-in-xamarinios.html, very curious.
This is the result of $PSVersionTable in PowerShell Integrated Console of vscode:
Name Value
PSVersion 5.1.19041.1023
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.19041.1023
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
Maybe same problem here:
pnp/powershell#727
I have removed module "PNP.PowerShell" in Version 1.7.0 and installed this module in version 1.5.0.
The error went away.
Then I updated module "PNP.PowerShell" back to Version 1.7.0 - the error was there again.
This can be reproduced.
Conclusion:
The problem must have something to do with module PNP.PowerShell Version 1.7.0.
But strange anyway: The exception is only thrown when running CMDLets in VSCode-PowerShell.
I will post an issue in PNP.PowerShell.
Going to reopen the issue here since it's quite possible this is at least partially an issue in PSES
Just for reference, the module sources: https://github.com/pnp/powershell
@Tom-CK Can you check if this works with PowerShell 7? We have a theory right now...
Hey @erwinvanhunen, this bug appears between 1.5.0 and 1.7.0, and we think it was the introduction of your own ALC that may have caused this unfortunate interaction. PSES's assembly load logic when running with Windows PowerShell (so full .NET) does not check for the assembly version, it only resolves by name, and we think that's the issue. Since PSES has its own resolve assembly event, the one in pnp/powershell@fd65170#diff-05a628297ec69a1e93f2c84d148c6fac638d6499eda9b04d32a914b628fbe660 isn't being fired. We fear that a lot might break if we change our assembly loader to start checking for name + version. 🧐
It looks like PnP also does some dependency wrangling and registers their own assembly resolve event, but neither of us checks the required assembly version.
I've had a go implementing that. Give the extension here a try and let me know what happens.
I loaded the zip, disabled the original powershell extension and it did not fix the issue for me. My issue might be different because so far I am only having issues using the "Teams" commands. The SharePoint commands I have used so far have not caused me any issue.
PS> code --list-extensions --show-versions | Select-String powershell
ms-vscode.powershell@2021.8.2
ms-vscode.powershell-preview@2021.8.2
PS> connect-pnponline -Url "https://m365x167780.sharepoint.com/sites/<site>"
-interactive
PS> Get-PnPTeamsTeam
PS>
The return should have been a list of 20-30 sites, I get nothing in the integrated console.
My issue might be different because so far I am only having issues using the "Teams" commands. The SharePoint commands I have used so far have not caused me any issue.
Well, that's good to know. 🤔
The return should have been a list of 20-30 sites, I get nothing in the integrated console.
Ok that's much stranger than getting an exception, and harder to debug unfortunately
The return should have been a list of 20-30 sites, I get nothing in the integrated console.
Ok that's much stranger than getting an exception, and harder to debug unfortunately
Here is the error when I try to target a specific site as this actually gives an error.
PS> Connect-PnPOnline -url "https://<tenant>.sharepoint.com/teams/TLTestTeam" -Interactive
PS> Get-PnPTeamsTeam -Identity "TLTestTeam"
Get-PnPTeamsTeam : Team not found
At C:\Users\tleduc\OneDrive - City of Kelowna\SharePoint Online\scripts\test2.ps1:2 char:1
+ Get-PnPTeamsTeam -Identity "TLTestTeam"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (:) [Get-PnPTeamsTeam], PSArgumentException
+ FullyQualifiedErrorId : PnP.PowerShell.Commands.Graph.GetTeamsTeam
PS> $error[0] | fl * -force
writeErrorStream : True
PSMessageDetails :
Exception : System.Management.Automation.PSArgumentException: Team not found
TargetObject :
CategoryInfo : ObjectNotFound: (:) [Get-PnPTeamsTeam], PSArgumentException
FullyQualifiedErrorId : PnP.PowerShell.Commands.Graph.GetTeamsTeam
ErrorDetails :
InvocationInfo : System.Management.Automation.InvocationInfo
ScriptStackTrace : at <ScriptBlock>, C:\Users\tleduc\OneDrive - City of Kelowna\SharePoint Online\scripts\test2.ps1: line 2
PipelineIterationInfo : {0, 1}
Sorry for my late reply, I was on holidays for some days:
Please remind:
- the error is thrown only when working VSCode-PowerShell. It it not thrown when using regular PowerShell.
- after installing package PNP.PowerShell in Version 1.5.0 everything worked fine. Went back to 1.7.0 the error was thrown
in VSCode PowerShell-Extension only.
Sorry for my late reply, I was on holidays for some days:
Please remind:
- the error is thrown only when working VSCode-PowerShell. It it not thrown when using regular PowerShell.
- after installing package PNP.PowerShell in Version 1.5.0 everything worked fine. Went back to 1.7.0 the error was thrown
in VSCode PowerShell-Extension only.
For me the error only exists in the "Powershell Integrated Console" terminal, switching terminals to "Powershell" I can run the commands and they work.
Yes 1.5.0 is the last working version for this issue.
@Tom-CK Can you try the dev build of the extension that Rob provided in this comment? #3510 (comment)
Your issue seems to be around the assembly loading, which we're trying to test.
@TobyLeduc the "PowerShell Integrated Console" is the extension's console, so that makes sense. The "powershell" terminal is just a hosted PowerShell terminal inside VS Code, it's not linked to the extension at all. So that does imply the issue is some incompatibility between PNP.PowerShell 1.7.0 and the extension itself.
@TobyLeduc for that error you posted, what's the result of $error[0].Exception | fl * -force?
@TobyLeduc for that error you posted, what's the result of
$error[0].Exception | fl * -force?
PS> $error[0].Exception | fl * -force
ErrorRecord : Team not found
Message : Team not found
ParamName :
Data : {}
InnerException :
TargetSite :
StackTrace :
HelpLink :
Source :
HResult : -2147024809
@TobyLeduc Could there be some missing state that you're relying on? Because that error looks legitimate, that the command executed but the team was not found.
@TobyLeduc Could there be some missing state that you're relying on? Because that error looks legitimate, that the command executed but the team was not found.
I don't think so. Even if I just request all teams I get zero results and no errors, yet I know there are about 100 teams in the tenant. It only appears to be an issue when using the integrated console and pnp.powershell > 1.5
Connect-PnPOnline -url "https://<tenant>.sharepoint.com" -Interactive
Get-PnPTeamsTeam
I have even tried the same two lines on a different system (server 2012r2) with a first time install of vscode/pnp.powershell. I have also switched to a Contoso test tenant to rule out potential issues with our tenant.
Ok, I definitely thinks its an issue with both PSES and PNP.PowerShell using their own Assembly Load Contexts.
I'm unable to login:
Connect-PnPOnline -Url "https://xxxx.sharepoint.com/sites/IT-InfrastrukturundSupport" -Credentials (Get-Credential) -CreateDrive -DriveName "SPO"
Connect-PnPOnline: Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.11.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Could not find or load a specific file. (0x80131621)
$error[0].Exception | fl * -force
Message : Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.11.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Could not find or load a specific file. (0x80131621)
FileName : System.IdentityModel.Tokens.Jwt, Version=6.11.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
FusionLog :
TargetSite : PnP.PowerShell.Commands.Base.PnPConnection CreateWithCredentials(System.Management.Automation.Cmdlet, System.Uri, System.Management.Automation.PSCredential, Boolean, System.String,
PnP.Framework.AzureEnvironment, System.String, System.String, Boolean, PnP.PowerShell.Commands.Enums.InitializationType)
StackTrace : at PnP.PowerShell.Commands.Base.PnPConnection.CreateWithCredentials(Cmdlet cmdlet, Uri url, PSCredential credentials, Boolean currentCredentials, String tenantAdminUrl, AzureEnvironment
azureEnvironment, String clientId, String redirectUrl, Boolean onPrem, InitializationType initializationType)
at PnP.PowerShell.Commands.Base.ConnectOnline.ConnectCredentials(PSCredential credentials, InitializationType initializationType) in
c:\PnPPowerShell\src\Commands\Base\ConnectOnline.cs:line 512
at PnP.PowerShell.Commands.Base.ConnectOnline.Connect(CancellationToken& cancellationToken) in c:\PnPPowerShell\src\Commands\Base\ConnectOnline.cs:line 258
at PnP.PowerShell.Commands.Base.ConnectOnline.ProcessRecord() in c:\PnPPowerShell\src\Commands\Base\ConnectOnline.cs:line 208
at System.Management.Automation.CommandProcessor.ProcessRecord()
Data : {TimeStampUtc}
InnerException : System.IO.FileLoadException: Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.11.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)
at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at System.Reflection.Assembly.LoadFromResolveHandler(Object sender, ResolveEventArgs args)
at System.Runtime.Loader.AssemblyLoadContext.InvokeResolveEvent(ResolveEventHandler eventHandler, RuntimeAssembly assembly, String name)
HelpLink :
Source : PnP.PowerShell
HResult : -2146232799
There is no issue with Windows Powershell (aka 5.1) or Powershell 7.1!
Same issue with PnP.Powershell 1.5.0
WARNING:
A newer version of PnP PowerShell is available: 1.7.0.
Use 'Update-Module -Name PnP.PowerShell ' to update.
Use 'Get-PnPChangeLog -Release 1.7.0' to list changes.
You can turn this check off by setting the 'PNPPOWERSHELL_UPDATECHECK' environment variable to 'Off'.
Connect-PnPOnline: Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.8.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Could not find or load a specific file. (0x80131621)
$error[0].Exception | fl * -force
Message : Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.8.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Could not find or load a specific file. (0x80131621)
FileName : System.IdentityModel.Tokens.Jwt, Version=6.8.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
FusionLog :
TargetSite : PnP.PowerShell.Commands.Base.PnPConnection CreateWithCredentials(System.Management.Automation.Cmdlet, System.Uri, System.Management.Automation.PSCredential, Boolean, System.String,
PnP.Framework.AzureEnvironment, System.String, System.String, Boolean, PnP.PowerShell.Commands.Enums.InitializationType)
StackTrace : at PnP.PowerShell.Commands.Base.PnPConnection.CreateWithCredentials(Cmdlet cmdlet, Uri url, PSCredential credentials, Boolean currentCredentials, String tenantAdminUrl, AzureEnvironment
azureEnvironment, String clientId, String redirectUrl, Boolean onPrem, InitializationType initializationType)
at PnP.PowerShell.Commands.Base.ConnectOnline.ConnectCredentials(PSCredential credentials, InitializationType initializationType) in
c:\PnPPowerShell\src\Commands\Base\ConnectOnline.cs:line 502
at PnP.PowerShell.Commands.Base.ConnectOnline.Connect(CancellationToken& cancellationToken) in c:\PnPPowerShell\src\Commands\Base\ConnectOnline.cs:line 257
at PnP.PowerShell.Commands.Base.ConnectOnline.ProcessRecord() in c:\PnPPowerShell\src\Commands\Base\ConnectOnline.cs:line 207
at System.Management.Automation.CommandProcessor.ProcessRecord()
Data : {TimeStampUtc}
InnerException : System.IO.FileLoadException: Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=6.8.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)
at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at System.Reflection.Assembly.LoadFromResolveHandler(Object sender, ResolveEventArgs args)
at System.Runtime.Loader.AssemblyLoadContext.InvokeResolveEvent(ResolveEventHandler eventHandler, RuntimeAssembly assembly, String name)
HelpLink :
Source : PnP.PowerShell
HResult : -2146232799
@rjmholt after following up your answer in the PowerShell Discord Server I created a "not-so" minimal repro example.
https://github.com/lucahost/PSES-ALC-Repro
as mentioned in the .\Run.ps1 script, running this with PowerShell 5.1 should work where running it with PSES will result in Errors
If I can provide anything more or something is not working I'm here to assist.
If you attach VisualStudio to the PowerShell Process (Set a Breakpoint before Import-Module) and go to Debug -> Windows -> Modules, you'll see from where the Assemblies are loaded.
@KoenZomers and @gautamdsheth are you two also looking at this? It seems to have been introduced when a custom ALC with a resolver was added to PNP.PowerShell, and it's not expecting another ALC (the one in Editor Services) to exist and possibly resolve dependencies. But we're still not entirely sure.
Not entirely sure here what I would need to do but I cannot reproduce this on my end.
What do I need to do differently to reproduce the issue?
@KoenZomers We've managed to narrow it down to reproducing with 1.7.0 and above, but not with 1.5.0. Which version did you test there?
@rjmholt after following up your answer in the PowerShell Discord Server I created a "not-so" minimal repro example. https://github.com/lucahost/PSES-ALC-Repro as mentioned in the .\Run.ps1 script, running this with PowerShell 5.1 should work where running it with PSES will result in Errors If I can provide anything more or something is not working I'm here to assist.
If you attach VisualStudio to the PowerShell Process (Set a Breakpoint before Import-Module) and go to Debug -> Windows -> Modules, you'll see from where the Assemblies are loaded.
@KoenZomers It looks like @lucahost has a full repro here.
This goes beyond my knowledge. We would need Erwin van Hunen to have a look at this, but he's currently overly occupied so that's likely going to take a while.
Good to know, thank you. Can we ping his handle for him to look at this when he eventually gets time?
I've added it to the list to discuss with him during our next meeting. Can someone that can reproduce this issue also try it with PnP PowerShell 1.6.0? And also with the latest prerelease build? Perhaps through narrowing down exactly in which version something changed we can look at the differences done in that release and figure it out.
@KoenZomers the problem is not per-se with PnP.PowerShell but with the ALC that resolves the DLL's.
There is a conflict when PowerShell Editor Services (Integrated Terminal) and the 3rd Party Package (PnP.PowerShell or my Repro Package (which includes EntityFramework Core as dependency)) are targeting other version of DLL's and both are using their own ALC.
For example:
- VS Code Editor Services is depending on Microsoft.Bcl.AsyncInterfaces version 5.0.0
- 3rd-party Package is depending on Microsoft.Bcl.AsyncInterfaces with version 1.1.1
Because VS Code PowerShell Editor already loaded their dependencies and has a ALC which does not check the requested dependency's version, it will return the already loaded version. So in the case of the example, when our code at some point requests Microsoft.BCL.AsyncInterfaces 1.1.1, we get 5.0.0.
In PnP.PowerShell, the dependency versions are most of the time equal to VS Code PowerShell Editor Services. So in fact, you don't see this exception very often.
cc @andschwa
Re-reading through this whole issue, please correct me if I'm wrong, but I believe these problems are only occurring with Windows PowerShell (e.g. PowerShell 5.1), not PowerShell Core (e.g. PowerShell 7). This makes some sense too given that it's an assembly loading issue and the ALCs used to separate dependencies from each other are no present in .NET Framework (which Windows PowerShell is bound to). So Windows PowerShell is loading the library from the GAC, and then when PnP.PowerShell needs a different version of the library, it cannot get it due to the fundamental nature of .NET Framework's assembly loading logic and Global Assembly Cache.
Because VS Code PowerShell Editor already loaded their dependencies and has a ALC which does not check the requested dependency's version, it will return the already loaded version. So in the case of the example, when our code at some point requests Microsoft.BCL.AsyncInterfaces 1.1.1, we get 5.0.0.
This is almost correct, except we tested that in #3510 (comment). This points to the issue being similar, but what I just outlined: the ALC logic is being avoided entirely due to this being an issue with Windows PowerShell 5.1 which uses .NET Framework and the GAC. The assembly resolve event handler in Windows PowerShell loads a version of Microsoft.BCL.AsyncInterfaces, and then because of the GAC, this is later returned to PnP.PowerShell. I'm unsure if there's a way around this, because the way around it is to use Assembly Load Contexts, but that's unavailable in this scenario.
@andschwa for me, this is still an issue with PS 7.1.2 using PSIC with the latest preview, the pop-up behavior for -useweblogin and -interactive never shows, however if I open a separate standalone PWSH terminal in vscode, works fine. Maybe something (or something missing) in the PSIC host implementation that is preventing it. Most other functions seem to work OK.
@JustinGrote, this is unfortunately out of my hands. We're waiting on a fix from the PnP.PowerShell team. @KoenZomers said they will have Erwin van Hunen take a look "soon."
We have discussed this, but turns out its very complex to resolve, if possible at all. In all honesty, with everything we have open and the limited resources we have contributing to the code at the moment, I would not count on this to be fixed soon if at all.
I would suggest anyone running into this use PowerShell Core which effectively (and correctly) avoids this problem by using an isolated assembly load context. That is the solution, it just cannot technically be implemented for Windows PowerShell because of limitations the Desktop .NET Framework.
I am using PowerShell Core and I am getting this issue
save-module ExchangeOnlineManagement -Path .\;
save-module pnp.powershell -path .\;
Import-Module .\PnP.PowerShell\1.10.0\PnP.PowerShell.psd1;
Import-Module .\ExchangeOnlineManagement\2.0.5\ExchangeOnlineManagement.psd1;
$runAs = [PSCustomObject]@{
"ClientId" = "xxx"
"Thumbprint" = "xxx"
"Tenant" = "xxx"
"Organization" = "xxx"
};
Connect-ExchangeOnline -AppId $RunAs.ClientId -CertificateThumbprint $RunAs.Thumbprint -Organization $RunAs.Organization -ErrorAction Stop -ShowBanner:$false;
Connect-PnPOnline -Url "xxx" -ClientId $RunAs.ClientId -Thumbprint $RunAs.Thumbprint -Tenant $RunAs.Tenant -ReturnConnection -ErrorAction Stop;
The ExchangeOnlineManagement loads a Microsoft.Identity.Client, Version=4.30.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae and PnP.PowerShell is expecting Microsoft.Identity.Client, Version=4.36.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae. I thought PowerShell Core has some fancy way of isolating the binaries
I wonder if we can "fix" this by having PnP.PowerShell update their dependency on Microsoft.Bcl.AsyncInterfaces.../cc @SeeminglyScience
Using Windows, PowerShell 7.3.2 and Visual Studio Code 1.74.3 and PowerShell for Visual Studio Code v2022.11.0 and also tried v2023.1.0. PnP.PowerShell 1.12.0.
Running code in Visual Studio Code, sometimes my script works (posted below), but most often I get this error:
Connect-PnPOnline: Could not load file or assembly 'Microsoft.Identity.Client, Version=4.36.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae'. Could not
find or load a specific file. (0x80131621).
In above posts it sounds like this cannot be fixed. I'm new to PowerShell 7 and Visual Code. Is there a workaround? I'm starting to get used to Visual Studio Code, but should I go back to the Windows PowerShell ISE?
##code is below
Install-Module -Name Microsoft.PowerShell.SecretManagement, Microsoft.PowerShell.SecretStore
Get-Secret -Name 'PS PnP'
Get-SecretInfo -Name 'PS PnP' | Select-Object -ExpandProperty Metadata
$certificatePassword = Get-Secret -Name 'PS PnP'
$siteurl = "https://XXXX.sharepoint.com/sites/yyyy"
$listname= "User Setup"
Connect-PnPOnline -Url $siteurl
-Tenant XXXX.onmicrosoft.com -ClientId "xxxxxxxx-yyyy-zzzz-aaaa-bbbbbbbbbbbb"
-CertificatePath "C:\Data\PS\creds\BBBB PnP.pfx" `
-CertificatePassword $certificatePassword
I will try to spend the next community day (Monday) looking at this again, though I'm not hopeful. I think it's on PnP.PowerShell to either implement or fix their own Assembly Load Context.
@andschwa - thank you taking the time to investigate it. We have launched a new 2.x nightly which has dropped support PS5 and is based completely on .NET 6 , maybe it can help ?
The nightly builds are available via PSGallery and using -Allowprerelease parameter to install the module
@gautamdsheth that sounds super promising!
@gautamdsheth have you seen pnp/powershell#1814 (reply in thread)? I'm going to see if we load Microsoft.Identity.Client.dll but that comment sure seems to be making a lot of sense.
Just dumping some info here, I can confirm that PnP.PowerShell (both 2.x and 1.x) depend on older versions of Microsoft.Identity.Client than requested by ExchangeOnlineManagement and MicrosoftTeams modules. And so far as I've been able to dig, none of PowerShell Editor Services, Secret Management, nor Secret Store depend on this module.
@gautamdsheth is there a way to get PnP.PowerShell to accept 4.0 and up? I think you might have some "must be this version" logic that's causing the 4.29 from Teams and the 4.44 from Exchange to be rejected, but I'm willing to bet that if it did allow it to be used, it might just work fine. I think that's something to do with bindingRedirects right?
Considering people are having issues with PowerShell 7 and only in the Extension Terminal, where we have an ALC, I'm thinking Rob may have been onto something when he surmised that it's because we don't check assembly versions in PSES. But this is far outside my expertise, I'll chat with @SeeminglyScience.
@andschwa - Yes, we are going to bump the MSAL.NET to the latest version , mostly 4.49 as part of the update to v2. Hopefully, will fix things.
@gautamdsheth have you seen pnp/powershell#1814 (reply in thread)? I'm going to see if we load
Microsoft.Identity.Client.dllbut that comment sure seems to be making a lot of sense.
Hello @andschwa & @gautamdsheth
I posted a similar message 4 days ago here, I think it went unnoticed because it is not an issue :
I hope version 2 will be able solve this problem... It's really annoying to have module cohabitation problems because of this DLL.
Since you now have a .net 6 only version, you really should look into using assemblyLoadContext to abstract your dependencies from the PowerShell shared assembly space to avoid this problem.
https://github.com/rjmholt/ModuleDependencyIsolationExample
EDIT: Sorry this was directed at PnP development, I forgot what repo I was in, oops.
@JustinGrote thing is, they are using an ALC. Somehow the combination of being loaded in the extension terminal where PSES is using an ALC breaks their ALC. Yes, it should "just be working" because of module dependency isolation...but for reasons I don't know where to begin to find out, it's not.
Oh, ohhh, oh! @gautamdsheth I was finding PnP.PowerShell's ALC to link here again, which is nearly an exact copy of the example from Rob that @JustinGrote just shared...and with a fresh set of eyes and a cup of coffee in me, I think I spied the issue:
protected override Assembly Load(AssemblyName assemblyName)
{
string assemblyFileName = $"{assemblyName.Name}.dll";
// Make sure we allow other common PowerShell dependencies to be loaded by PowerShell
// But specifically exclude Microsoft.ApplicationInsightssince we want to use a different version here
if (!assemblyName.Name.Equals("Microsoft.ApplicationInsights", StringComparison.OrdinalIgnoreCase))
{
string psHomeAsmPath = Path.Join(s_psHome, assemblyFileName);
if (File.Exists(psHomeAsmPath))
{
// With this API, returning null means nothing is loaded
return null;
}
}
// Now try to load the assembly from the dependency directory
string dependencyAsmPath = Path.Join(_dependencyDirPath, assemblyFileName);
if (File.Exists(dependencyAsmPath))
{
return LoadFromAssemblyPath(dependencyAsmPath);
}
return null;
}My understanding of this is that Microsoft.ApplicationInsights is the only dependency being loaded from the module's dependency directory path, everything else (such as Microsoft.Identity.Client, the problem) is being loaded from the shared path of PowerShell (or from a previous resolution).
I think you are quite close to the issue @andschwa, the problem occurs, if shared dependencies exist in s_psHome which are requested. Then they are returned / loaded from PnP's ALC even if the requested Version does not match.
I'm not so sure, but a Version-Check as in PsesLoadContext could do the trick.
For dependencies not existing in s_psHome the code is partially working, they are loaded from the modules-directory but could have the wrong version as well.
This issue has been marked as external. It has been automatically closed for housekeeping purposes.
@lucahost I think you are right and that it's the version check that's missing, presumably causing an the older version of Microsoft.Identity.Client shipped with PnP.PowerShell to be returned as resolved even when the requestor requires a newer version.
It's actually worse than this. The AssemblyLoadContext as implemented in PnP.PowerShell is only isolating Microsoft.ApplicationInsights, and PnP.PowerShell has a lot more dependencies (namely Microsoft.Identity.Client) that conflict and need to be isolated in the same manner.
any suggestions or workarounds for this?
This issue seems to be going for a years and I stucked with my development with this
I can do nothing in VS Code which is the one IDE I can use for Powershell 7.x
You can use a regular pwsh terminal in VS Code, just not the "Extension Terminal." Click the + button to spawn a new terminal in the pane.
You can use a regular
pwshterminal in VS Code, just not the "Extension Terminal." Click the+button to spawn a new terminal in the pane.
Thank you, this workaround works for me