Azure/ARO-RP

Error: The resource provider does not have Network Contributor permission on vnet

akinfemi opened this issue · 6 comments

akinfemi commented

Trying to deploy an ARO 4.5 Cluster through the ARM template but get this error:

Error: Error waiting for deployment: Code="DeploymentFailed" Message="At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details." Details=[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"InvalidResourceProviderPermissions\",\r\n        \"message\": \"The resource provider does not have Network Contributor permission on vnet '/subscriptions/xxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/resourceGroups/<ResourceG>/providers/Microsoft.Network/virtualNetworks/vnet'.\"\r\n      }\r\n    ]\r\n  }\r\n}"}]

NOTE:

  • The Service Principal used have User Access and Contributor roles, with Subscription level scope.
  • This error only occurs when using ARM template i.e using the az aro create --client-id xxx --client-secret works fine.

I also tried adding the Network Contributor role specifically to the SP, that didn't work either.

Template:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "location": {
            "type": "string"
        },
        "azClientId": {
            "type": "string"
        },
        "azClientSecret": {
            "type": "string"
        },
        "clusterName": {
            "defaultValue": "arocluster",
            "type": "string"
        },
        "apiServerVisibility": {
            "type": "string"
        },
        "ingressVisibility": {
            "type": "string"
        },
        "virtualNetworkCIDR": {
            "type": "string"
        },
        "masterSubnetID": {
            "type": "string"
        },
        "workerSubnetID": {
            "type": "string"
        },
        "domain": {
            "type": "string"
        },
        "masterVmSize": {
            "type": "string",
            "defaultValue": "Standard_D8s_v3"
        },
        "workerVmSize": {
            "type": "string",
            "defaultValue": "Standard_D16s_v3"
        },
        "workerVmCount": {
            "type": "string",
            "defaultValue": "3"
        },
        "workerVmDiskSize": {
            "type": "string",
            "defaultValue": "128"
        },
        "resourceGroupId": {
            "type": "string"
        }
    },
    "variables": {
        "serviceCidr": "192.30.0.0/16",
         "podCidr": "10.128.0.0/14"
    },
    "resources": [
        {
            "type": "Microsoft.RedHatOpenShift/openShiftClusters",
            "apiVersion": "2020-04-30",
            "name": "[parameters('clusterName')]",
            "location": "[parameters('location')]",
            "properties": {
                "clusterProfile": {
                    "domain": "[parameters('domain')]",
                    "resourceGroupId": "[parameters('resourceGroupId')]"
                },
                "servicePrincipalProfile": {
                    "clientId": "[parameters('azClientId')]",
                    "clientSecret": "[parameters('azClientSecret')]"
                },
                "networkProfile": {
                    "podCidr": "[variables('podCidr')]",
                    "serviceCidr": "[variables('serviceCidr')]"
                },
                "masterProfile": {
                    "vmSize": "[parameters('masterVmSize')]",
                    "subnetId": "[parameters('masterSubnetID')]"
                },
                "workerProfiles": [
                    {
                        "name": "worker",
                        "vmSize": "[parameters('workerVmSize')]",
                        "diskSizeGB": "[int(parameters('workerVmDiskSize'))]",
                        "subnetId": "[parameters('workerSubnetID')]",
                        "count": "[int(parameters('workerVmCount'))]"
                    }
                ],
                "apiserverProfile": {
                    "visibility": "[parameters('apiServerVisibility')]"
                },
                "ingressProfiles": [
                    {
                        "name": "default",
                        "visibility": "[parameters('ingressVisibility')]"
                    }
                ]
            }
        }
    ]
}

Are there other required permissions not listed in the documentation?

olga-mir commented

@akinfemi In addition to cluster Service Principal, you also need to create Service Principal for the Resource Provider. RP needs Network Contributor permissions on the vnet.
As far as I know we do not have official documentation for ARM deployment method, the best resource I can point at this stage is this repository created by a Microsoft Cloud team member: https://github.com/jmo808/arm-aro43.
Hope this helps

  • 👍1
akinfemi commented

Yes, assigning the Network Contributor role to the RP object ID helped. Thanks

apsessoms commented

Yes, assigning the Network Contributor role to the RP object ID helped. Thanks

Can you expand on what exactly this means? Assigning a service principal for the resource provider? My team and I can't figure this one out. Having the same issue.

akinfemi commented

Yes, assigning the Network Contributor role to the RP object ID helped. Thanks

Can you expand on what exactly this means? Assigning a service principal for the resource provider? My team and I can't figure this one out. Having the same issue.

Get ARO's Resource Provider SP following these instructions from the readme:

The ARM template also needs to grant the ARO 4 Resource Provider service principal permissions in order to provision and manage clusters. To obtain the ARO 4 RP service principal object id execute the following command.

az ad sp list --filter "displayname eq 'Azure Red Hat OpenShift RP'" --query "[?appDisplayName=='Azure Red Hat OpenShift RP'].{name: appDisplayName, objectId: objectId}"

Assign the role. Here's the code block in the ARM template: [L214-L225](https://github.com/jmo808/arm-aro43/blob/master/azuredeploy.json#L214C9-L225C11)