Azure/ARO-RP

ARO - Bring your own resource group gives an error

abymsft opened this issue · 4 comments

abymsft commented

Hi,

I am getting an error when I provision an ARO cluster in a new resource group (rg-aro-test) when the VNET+subnets have already been provisioned in a separate resource group (rg-os-cluster). The switch --cluster-resource-group returns
Invalid --cluster-resource-group 'rg-aro-test': resource group must not exist.

Can this be fixed?

Additionally, when using --vnet switch with az aro create, it seems that the ARO cluster can only be created in the resource group where the VNET resources reside. Otherwise, I get an error
(ResourceNotFound) The Resource 'Microsoft.Network/virtualNetworks/os-vnet' under resource group 'rg-aro-test' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix

Will this be fixed? This doesn't sit well with having VNets and other common shared services in a separate resource group

Steps to reproduce

  1. Create two Resource Groups
    rg-os-common (used for common resources needed for ARO clusters such as vNet etc.)
    rg-aro-test (where I expect the ARO cluster to get provisioned)

  2. Create VNet
    az network vnet create --resource-group rg-os-cluster --subscription <subid> --name os-vnet --address-prefixes 10.0.0.0/22

  3. Create MasterSubnet
    az network vnet subnet create --resource-group rg-os-cluster --subscription <subid> --vnet-name os-vnet --name master-subnet --address-prefixes 10.0.0.0/23 --service-endpoints Microsoft.ContainerRegistry

  4. Create WorkerSubnet
    az network vnet subnet create --resource-group rg-os-cluster --subscription <subid> --vnet-name os-vnet --name worker-subnet --address-prefixes 10.0.2.0/23 --service-endpoints Microsoft.ContainerRegistry

  5. Update MasterSubnet
    az network vnet subnet update --name master-subnet --resource-group rg-os-cluster --<subid> --vnet-name os-vnet --disable-private-link-service-network-policies true

  6. Create the ARO cluster az aro create --name aroclusterone --vnet os-vnet --master-subnet master-subnet --worker-subnet worker-subnet --cluster-resource-group rg-aro-test
    Gives an error
    Invalid --cluster-resource-group 'rg-aro-test': resource group must not exist.

gvanderpotte commented

Hello,

Can this be fixed?

value passed as parameter --cluster-resource-group is expected to be a non-existing resource group. ARO CLI will use this parameter to name the resource group that will hold your cluster resources (VMs etc). If not provided, it will use a random generated name for that resource group. ARO CLI is working as expected here.

Additionally, when using --vnet switch with az aro create, it seems that the ARO cluster can only be created in the resource group where the VNET resources reside. Otherwise, I get an error

Can you please elaborate on that and give the exact command that was used to led to this error ?

abymsft commented

@gvanderpotte regarding the commands on the vnet issue please check the "Steps to reproduce" section.

The use case as I mentioned
"
is creating an ARO cluster in a resource group that already exists with VNETs+Subnets. This works for scenarios where in a resource group the customer lets assume uses Azure Web AppService today but in future wants to use Azure RO. Since the Vnets and other resources are already provisioned in this rg the customer wants to be able to provision an ARO cluster inside it instead of hosting the ARO cluster in a new resource group.
"

Hence the need of tweaking the behavior of --cluster-resource-group so that it takes the name of an existing resource group

Additionally, the cli reference for --cluster-resource-group here is unclear and does not mention the intent of the swtich.

gvanderpotte commented

Hello @abymsft

  1. There are no plans to have --cluster-resource-group switch behave differently and let a non-empty resource group be passed here. The intent is that ARO RP needs to have full control of the cluster-resource-group and hence such rg cannot be brought by customer.
    Documentation will be updated to ensure the purpose of this switch is clarified

  2. Regarding the question around the vnet resource group, if you specify --vnet name which is not a resourceID but a simple name, and if the --vnet-resource-group is not specified, aro cli will consider that the vnet is in the same rg as specified in -resource-group parameter. If it is not case, it will throw an error. If the vnet rg is different from the one specified in --resource-group, then --vnet-resource-group needs to be specified (or vnet be a fully qualified resource ID). This is why I first asked the full command on this question, cause the --resource-group parameter was not visible on it (neither was vnet-resource-group).

Regards

abymsft commented

Thanks @gvanderpotte . Ok to close this issue.