In this article I am going to show you how to build a Role Based Access Controlled (RBAC) Azure Kubernetes Services (AKS) cluster using Terraform and Azure Active Directory. At the time of writing this article, when you create an AKS cluster using the portal or terraform RBAC is disabled by default. Luckily since version 1.19.0 of the AzureRM Terraform provider RBAC is supported.
You can find all the files used at the following GitHub repository.
Before you can set up your new AKS cluster you need to make sure you have terraform installed on your local machine and it set up correctly. You can find out how to do that using this guide. https://learn.hashicorp.com/terraform/getting-started/install.html
You will also need a Service Principal. You can read my article, First look at terraform (https://pixelrobots.co.uk/2018/11/first-look-at-terraform-and-the-azure-cloud-shell/) to get this. Make sure you take note of the App ID (Client ID) and Password (Client Secret) , we will need them for the variables.tf file later.
An Azure Storage account with a container and an Access Key to store your Terraform state file. How to do this using this guide from Microsoft. https://pixelrobots.co.uk/2019/01/how-to-store-your-terraform-state-file-in-azure-storage/
An SSH certificate for the Linux VMs for your AKS cluster. You can read more about creating them here. https://docs.microsoft.com/en-us/azure/virtual-machines/linux/ssh-from-windows. Just make sure you have it saved in the same path that's stated in the variables terraform file.
AKS with RBAC needs two applications created in Azure AD. The first one is a Server application, the second is a client application. We will use the Azure portal to create them.
Note:
You can use the same Server application for multiple AKS clusters, but it is recommended to use one Client application per cluster.
This application is used to get a users Azure AD group membership.
In the Azure Portal navigate to Azure Active Directory and then click on App registrations and click New application registration .
In here we need to enter a Name and make sure the Application type is Web app / API. In the Sign-on URL enter any web address. I am using my domain name. Then click Create .
In the new blade click on Manifest .
In here we need to edit the groupMembershipClaims value to "All" . Make sure to include the ". Then click Save .
Now Click on Settings and then click on Keys .
Now enter a Description for the key and select when you would like it to Expire . Then click Save .
Take a copy of the Value . We will need it later when we create the AKS cluster. The value is referred to as the Server application secret .
Warning:
You will not be able to get this value again if you leave this blade. Make sure you copy it.
Now click on Required permissions In this blade click on + add .
Click Select an API then Microsoft Graph, then click Select .
Under Application permissions put a tick next to Read Directory Data .
Scroll down further to Delegated permissions . Under here put a tick next to Sign in and read user profile . Then click Select . In the next blade click Done.
Now we have to Grant admin consent . All we a have to do is click the button.
Click Yes.
Take a note of the Application ID we will need it for later.
This application is used when logging in using the Kubectl the Kubernetes CLI.
Navigate back to the Azure Active Directory blade again and click on App registrations . Create a new one again.
Enter a Name and then under Application type select Native. Add a Redirect URI again I have used my domain. Then click Create.
Now click on Settings and click Required permissions . In here click on Add.
Click on Select an API. In the search box enter the name of the Server application we just created. Click it and then click Select .
Put a tick next to Access AKSRBAC. (the AKSRBAC is your server application name.) Then click Select . IN the next blade click Done.
Now we have to Grant admin consent . All we a have to do is click the button and then click Yes.
Now take a note of the Application ID. This will be the Client application ID.
Get the Tenant ID
Now we need to get the Tenant ID. This is easy. Just go back to Azure Active Directory in the Azure portal and click on Properties . In here you will see the Tenant ID .
You should now have a set of IDs like the ones I do below.
Server application secret: rfHXIJmz6d9/sTHQk4ekyvescN7PcogFyIVmYytmxBs=
Server Application ID: c59c8bf4-c1be-46a5-992a-18efdd9b08ac
Client Application ID: 9418f3aa-7845-4de8-90bf-0231ad06450b
Tenant ID: d8171bb5-a0de-40a6-afdf-8b569cf6dbb8
Now its time to deploy the AKS cluster using terraform.
First we need to edit the variables.tf file from the GitHub repo with the right names and values for your environment. We will need to also add our IDs we have from above along with our Service Principal details.
Now its time to initialize Terraform. First, we need to update the backend.tfvars file with our storage account details for the tfstate file. You should have all this information if you followed the guide in the prerequisites. To actually initialize terraform in your VS Code Bash terminal or Windows subsystem for Linux terminal type the following. Just make sure you're in the directory with the terraform files.
You will need to login to your Azure subscription first use:
az login
terraform init -backend-config=backend.tfvars
Lets test our Terraform files to see what will happen. We use the plan option for this.
terraform plan -out "out.plan"
Everything looks good. 4 items are going to be created. Now its time to actually apply the configuration. To do that just run:
terraform apply "out.plan"
Its going to take some time to build everything. Maybe 20 minutes or more. You might want to go get a cup of tea.
That's the cluster deployed! Now its time for us to configure RBAC. To do this we need to create Cluster Role Binding and a Cluster Role using a yaml file. But first we need to connect to Kubernetes cluster as an admin. Use the following command to do that. Just change the resource group and name to match yours.
az aks get-credentials --resource-group pixelrobots-tst-aks --name pixelrobots-tst-aks --admin
In the Git repo under the k8s folder you will find two yaml files one to add a user the other for a group. The user one is easy. You just change the email address at the bottom. For the group one you will need to go into Azure AD and get the Group Object ID . Once you have the .yaml file you want to use ready. Make sure your in the directory with the files and then type the following to apply it.
kubectl apply -f rbac-aad-group.yaml
Now that we have configured the cluster for RBAC its time to connect to it. Lets get some non admin credentials first.
az aks get-credentials --resource-group pixelrobots-tst-aks --name pixelrobots-tst-aks
Lets use the kubectl to see what nodes we have.
kubectl get nodes
You will notice it is asking us to sign in to the azure portal. Go ahead and do it.
As you can see you can see the two nodes in the cluster. And that's it. You now have an RBAC AKS cluster. If you have any questions please reach out.
