Request was throttled - Failing
mla157 opened this issue · 4 comments
Hello AzureADAssessment-Team,
since the last versions I'm running in a issue in our production environment.
After a long runtime of 1h 50m there is multiple throttlling occuring:
WARNING: Using a confidential client is non-interactive and requires that the necessary scopes/permissions be added to the application or have permissions on-behalf-of a user.
WARNING: Using a confidential client is non-interactive and requires that the necessary scopes/permissions be added to the application or have permissions on-behalf-of a user.
WARNING: Using a confidential client is non-interactive and requires that the necessary scopes/permissions be added to the application or have permissions on-behalf-of a user.
WARNING: Request returned error and will attempt retry 1 of 5 after 30s.
WARNING: Request returned error and will attempt retry 2 of 5 after 60s.
WARNING: Request returned error and will attempt retry 3 of 5 after 120s.
WARNING: Request returned error and will attempt retry 4 of 5 after 240s.
WARNING: Request was throttled and will attempt retry 5 of 5 after 30s
And then comes the fail:
Catch-MsGraphError : This request is throttled. Please try again after the value (in seconds) specified in the Retry-After header. CorrelationId: 638b720b-aa65-4df0-852e-5bfd4d0dd708
At C:\Users\VssAdministrator\Documents\PowerShell\Modules\AzureADAssessment\2.3.11\internal\Get-MsGraphResults.ps1:484 char:49
+ Catch-MsGraphError $_
+ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (Method: GET, Reques…ssessment/2.3.11
}:HttpRequestMessage) [Invoke-RestMethod], HttpResponseException
+ FullyQualifiedErrorId : UnknownError,Catch-MsGraphError
##[error]PowerShell exited with code '1'
In smaller Environments (e.q. our test environment) I do not run into a issue.
I noticed that on older versions not so much throtteling was happening.
Maybe there is error in the retry logic.
Looking forward to an answer.
Can you share some metrics on the tenant you are running against? E.g. number of users, groups, devices and applications? You should be able to see these stats in the home page of the Azure AD portal. tx
Hey @merill,
I can share you some approximate values:
Users - 300.000
Groups - 100.000
Devices - 225.000
It looks like this was working previously for the same tenant size previously and the throttling is a new behaviour. We are following up with the service team to dig deeper into this.
In the meantime, would you be able to run this interactively with a user account (instead of a service principle) and let us know if you see the throttling issue? This will help us with the troubleshooting.