Processing Azure AS using Data Factory MSI
otykier opened this issue · 10 comments
In the readme, you mention that it should be possible to use the MSI of Data Factory instead of a service principal.
I tried creating a pipeline with a single Web Activity that uses MSI authentication to start the processing using the async API. I made sure that the application ID behind the MSI is added to the list of Analysis Services Administrators. For the "Resource" property of the Web Activity, I put https://*.asazure.windows.net.
Unfortunately, the activity fails without any error message. It seems like the MSI authentication feature of Web Activity does not work against the Async rest endpoint.
Next, I tried to see if I could obtain a token for the MSI principal manually in another Web Activity, before calling the Async API to start the processing. No matter what I do, though, it seems that I need to know the client secret, in order to obtain a token, and this is of course not possible for MSI.
Any insight you have would be greatly appreciated. Thanks!
Thanks for reporting Daniel. Honestly this is just me being lazy. I created this sample before MSI was supported on a Web Activity and haven’t gone back to test it. It worked for my Azure SQL DW samples so I assumed it would work here. I will put this on the list to look into and will get back to you.
Forget about this - I made a stupid mistake: When typing in the URL of the async REST API, I had asazure:// instead of https:// - after correcting this, it works like a charm with MSI authentication directly in the Web Activity. Nice!
No worries. This was a good impetus to me to get the MSI version working. I've published an MSI version now and documented it here:
https://github.com/furmangg/automating-azure-analysis-services/blob/master/README.md#processazureas
Also, I was able to simplify the MSI and non-MSI version so it doesn't require an HTTP linked service or dataset anymore. So that's nice it's just all in that one pipeline.
Hey! I guess this is related to the above
I'm using ProcessAzureAS MSI.json and followed the steps with adding the ADF MSI as an administrator of AAS.
The error I get running the pipeline is
{
"errorCode": "2108",
"message": "{"code":"Unauthorized","subCode":0,"message":"An internal error occurred.","timeStamp":"2019-05-27T15:22:09.5300402Z","httpStatusCode":400,"details":[{"code":"RootActivityId","message":"4b9e83b6-634e-4b19-9a58-a17606189c2f"},{"code":"Param1","message":"asazure://asazureweu3-westeurope.asazure.windows.net/[myresourcename]"}]}",
"failureType": "UserError",
"target": "StartProcessingAzureAS"
}
I also tried making it Contributor on the Azure resource, but no success.
Any idea of what is wrong?
@erikatgaia my suggestion would be that you screenshot everything you did and post it here so we can see if we spot a problem. Finding the right IDs is tricky
@furmangg I did as you described in the readme to find application id and directory id. And added it manually as a Server administrator in the Security tab for the AAS in Management Studio.
Actually, I also could search for the datafactory name in that same dialog also directly, and it was added with the same id.
What values did you use for the pipeline parameters? Particularly, what did you use for the Region parameter? ("The error message above said asazure://asazureweu3-westeurope.asazure.windows.net/[myresourcename] but I'm assuming the Region you used was "westeurope" alone?) Are you sure you got the TenantID and SubscriptionID parameters right?
Can you confirm that the StartProcessingAzureAS activity is setup like this?
Can you go to the Azure portal and find your ADF and look at the Properties tab. Can you confirm the Managed Identity Application ID and the Managed Identity Tenant was what you used?
The Managed Identity Application ID under the Properties of the datafactory-resource was different to the Application ID I found in the AAD. Strange... when I tried the other ID it worked!
Very strange that they differ, do you have any clue why?
Yes, I use westeurope as a Region. The TenantID (which seems to be be same as DirectoryID) and SubscriptionID is correct, for some reason it's converted to this (asazure://asazureweu3-westeurope.asazure.windows.net/)
Very grateful for your help, Greg!
Erik
Glad it is working. I will update the instructions since that seems easier than the prior approach.
I can’t explain the discrepancy. Did you have multiple Azure resources with the same name (such as a VM with MSI with the same name as ADF)?
Well, I've noticed that if I search for my datafactory resource name in Add role assignment for the Access Control of another resource, the resource name appears two times. I don't know why, but I have something in my head that the people who set up this environment the first time did some mistake and needed to delete the resources and create them again in some way, not sure exactly how they did it
There are no other resource with the same name what I know, if I search for it under AAD -> Enterprise applications it only appears once. And also if I search the resources in the Azure portal main search