Azure/azure-functions-powershell-worker

Emerging Issue: Az-Accounts

andystaples opened this issue · 8 comments

andystaples commented

Due to an unexpected platform issue emerging in Azure Functions, some customers may begin to experience the following errors in their PowerShell applications when invoking the Connect-AzAccount commandlet, or potentially other commandlets from the Az.Accounts module. The exception will look something like this:
[Error] ERROR: Could not load file or assembly 'Microsoft.Identity.Client, Version=4.49.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae'. The system cannot find the path specified. Exception : Type : System.IO.FileNotFoundException Message : Could not load file or assembly 'Microsoft.Identity.Client, Version=4.49.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae'. The system cannot find the path specified.

A temporary workaround has been discovered while engineers work on a permanent solution:

  • If not already present, add the following somewhere early in function code, preferably in profile.ps1:

Import-Module Az.Accounts

  • Immediately after importing the module, add the following lines:
$azAccountsLibPath = Join-Path (Split-Path (Get-Module Az.Accounts).Path -Parent) "lib"
Add-Type -AssemblyName (Join-Path $azAccountsLibPath "netstandard2.0" "Microsoft.IdentityModel.Abstractions.dll")
Add-Type -AssemblyName (Join-Path $azAccountsLibPath "netcoreapp2.1" "Microsoft.Identity.Client.dll")
Add-Type -AssemblyName (Join-Path $azAccountsLibPath "netcoreapp3.1" "Microsoft.Identity.Client.Extensions.Msal.dll")

This will pre-load the offending binaries and prevent the exception when Connect-AzAccount is run.

VSaVen commented

If we apply this fix to profile.ps1, do we have to revert the changes later?

  • 👍1
SudarshanSMD commented

Faced this issue today on a Function App hosted in North Europe region with EPL1 plan.
The fix/hack works. Thank you.

Given the severity of the issue, the Platform update for the Azure functions should not have been pushed.

Kalsaleh22 commented

Hello everyone,

I have tried to add the commands to mitigate the issue on my function app that I have created with Az version 6, but I faced the below error:

**ERROR: Could not load file or assembly 'C:\home\data\ManagedDependencies\2311011228118745813.r\Az.Accounts\2.9.1\lib\netcoreapp2.1\Microsoft.Identity.Client.dll'. The system cannot find the file specified.

Exception :
Type : System.IO.FileNotFoundException
Message : Could not load file or assembly 'C:\home\data\ManagedDependencies\2311011228118745813.r\Az.Accounts\2.9.1\lib\netcoreapp2.1\Microsoft.Identity.Client.dll'. The system cannot find the file specified.
FileName : C:\home\data\ManagedDependencies\2311011228118745813.r\Az.Accounts\2.9.1\lib\netcoreapp2.1\Microsoft.Identity.Client.dll
TargetSite :
Name : LoadFromPath
DeclaringType : System.Runtime.Loader.AssemblyLoadContext
MemberType : Method
Module : System.Private.CoreLib.dll
Source : System.Private.CoreLib
HResult : -2147024894
StackTrace :
at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)
at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at Microsoft.PowerShell.Commands.AddTypeCommand.LoadAssemblies(IEnumerable`1 assemblies)
at Microsoft.PowerShell.Commands.AddTypeCommand.EndProcessing()
at System.Management.Automation.Cmdlet.DoEndProcessing()
at System.Management.Automation.CommandProcessorBase.Complete()
CategoryInfo : NotSpecified: (:) [Add-Type], FileNotFoundException
FullyQualifiedErrorId : System.IO.FileNotFoundException,Microsoft.PowerShell.Commands.AddTypeCommand
InvocationInfo :
MyCommand : Add-Type
ScriptLineNumber : 7
OffsetInLine : 1
HistoryId : 1
ScriptName : C:\home\site\wwwroot\CalculateStats\run.ps1
Line : Add-Type -AssemblyName (Join-Path $azAccountsLibPath "netcoreapp2.1" "Microsoft.Identity.Client.dll")

PositionMessage  : At C:\home\site\wwwroot\CalculateStats\run.ps1:7 char:1
                   + Add-Type -AssemblyName (Join-Path $azAccountsLibPath "netcoreapp2.1"  …
                   + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot     : C:\home\site\wwwroot\CalculateStats
PSCommandPath    : C:\home\site\wwwroot\CalculateStats\run.ps1
InvocationName   : Add-Type
CommandOrigin    : Internal

ScriptStackTrace : at , C:\home\site\wwwroot\CalculateStats\run.ps1: line 7**

I have upgraded the Az version to 10 then the issue resolved, but make sure to restart the function app after editing the requirements.psd1 file.

m-soltani commented

It seems that this is not limited to Microsoft.Identity Dll's, we started to see similar issue when we added Az.KeyVault entry to our requirement.ps1 file as follows:

@{
    'Az.Accounts' = '2.*'
    'Az.Network' = '5.*'
    'Az.ResourceGraph' = '0.11'
    'Az.Dns' = '1.*'
    'Az.Resources' = '6.*'
    'Az.Sql' = '4.*'
    'Az.KeyVault' = '4.*'
}

And the error log:


Result: ERROR: Could not load file or assembly 'Microsoft.ApplicationInsights, Version=2.18.0.315, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

Exception             : 
    Type       : System.IO.FileLoadException
    Message    : Could not load file or assembly 'Microsoft.ApplicationInsights, Version=2.18.0.315, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
    TargetSite : 
        Name          : Bind_LoadAssemblies
        DeclaringType : initialsessionstate
        MemberType    : Method
        Module        : System.Management.Automation.dll
    StackTrace : 
   at System.Management.Automation.Runspaces.InitialSessionState.Bind_LoadAssemblies(ExecutionContext context)
   at System.Management.Automation.Runspaces.InitialSessionState.Bind(ExecutionContext context, Boolean updateOnly, PSModuleInfo module, Boolean noClobber, Boolean local, Boolean setLocation)
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModuleManifest(String moduleManifestPath, ExternalScriptInfo manifestScriptInfo, Hashtable data, Hashtable localizedData, ManifestProcessingFlags manifestProcessingFlags, Version minimumVersion, Version maximumVersion, Version requiredVersion, Nullable`1 requiredModuleGuid, ImportModuleOptions& options, Boolean& containedErrors)
    Source     : System.Management.Automation
    HResult    : -2146232799
CategoryInfo          : InvalidOperation: (:) [Import-Module], FileLoadException
FullyQualifiedErrorId : FormatXmlUpdateException,Microsoft.PowerShell.Commands.ImportModuleCommand
InvocationInfo        : 
    MyCommand        : Import-Module
    ScriptLineNumber : 16
    OffsetInLine     : 5
    HistoryId        : 1
    ScriptName       : C:\home\site\wwwroot\profile.ps1
    Line             :     Import-Module Az.Accounts

    PositionMessage  : At C:\home\site\wwwroot\profile.ps1:16 char:5
                       +     Import-Module Az.Accounts
                       +     ~~~~~~~~~~~~~~~~~~~~~~~~~
    PSScriptRoot     : C:\home\site\wwwroot
    PSCommandPath    : C:\home\site\wwwroot\profile.ps1
    InvocationName   : Import-Module
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, C:\home\site\wwwroot\profile.ps1: line 16


Exception: Could not load file or assembly 'Microsoft.ApplicationInsights, Version=2.18.0.315, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
Stack:    at System.Management.Automation.Runspaces.InitialSessionState.Bind_LoadAssemblies(ExecutionContext context)
   at System.Management.Automation.Runspaces.InitialSessionState.Bind(ExecutionContext context, Boolean updateOnly, PSModuleInfo module, Boolean noClobber, Boolean local, Boolean setLocation)
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModuleManifest(String moduleManifestPath, ExternalScriptInfo manifestScriptInfo, Hashtable data, Hashtable localizedData, ManifestProcessingFlags manifestProcessingFlags, Version minimumVersion, Version maximumVersion, Version requiredVersion, Nullable`1 requiredModuleGuid, ImportModuleOptions& options, Boolean& containedErrors)

Is there any workaround for this case ?

PS: Looking at release notes: https://github.com/Azure/azure-powershell/blob/main/src/Accounts/Accounts/ChangeLog.md#version-2131 for Az.Accounts, the Microsoft.ApplicationInsights version was updated from 2.13.1 to 2.18.0 and you see the missing dll in the Error log above. So, another piece of information to consider.

  • 👍3
m-soltani commented

We were able to resolve the issue by pinning a specific version of the Az.KeyVault and Az.Accounts in requirements.ps1 which solved the module import problem.

StephenFerrero commented

@andystaples Any update on this? We are seeing the same with ApplicationInsights dependency on Az.Accounts 2.13.2. Is this the same issue?

Pinning an earlier version of Az.Accounts is problematic due to other Az and Microsoft.Graph module dependencies.

andystaples commented

The underlying issue has been patched in all regions for approximately 7 weeks now - any new issues should be considered unrelated and raised when appropriate to Microsoft Azure Support.
Apologies for the delay updating this thread.
/cc @StephenFerrero @m-soltani @Kalsaleh22

  • 👍2
RRanks commented

Problem is back... Looks like new updates may have thrown this issue back up?