terraform-azurerm-aks
Deploys a Kubernetes cluster on AKS with optional monitoring through Azure Log Analytics
This Terraform module deploys a Kubernetes cluster on Azure using AKS (Azure Kubernetes Service) and adds support for monitoring with Log Analytics.
-> NOTE: If you have not assigned client_id or client_secret, A SystemAssigned identity will be created.
Example usage in Terraform 1.0.0+
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "example" {
name = "aks-resource-group"
location = "eastus"
}
module "network" {
source = "Azure/network/azurerm"
resource_group_name = azurerm_resource_group.example.name
address_space = "10.0.0.0/16"
subnet_prefixes = ["10.0.1.0/24"]
subnet_names = ["subnet1"]
depends_on = [azurerm_resource_group.example]
}
data "azuread_group" "aks_cluster_admins" {
name = "AKS-cluster-admins"
}
module "aks" {
source = "Azure/aks/azurerm"
prefix = "prefix-${random_id.prefix.hex}"
resource_group_name = azurerm_resource_group.example.name
vnet_subnet_id = module.network.vnet_subnets[0]
sku_tier = "Paid"
enable_auto_scaling = true
min_count = 1
max_count = 2
node_count = 1
max_pods = 110
depends_on = [module.network]
node_labels = {
"node1" = "label1"
}
node_tags = {
"tag1" = "Tag1"
}
}
output "module_aks_kube_config_raw" {
value = module.aks[*].kube_config_raw
sensitive = true
}The module supports some outputs that may be used to configure a kubernetes provider after deploying an AKS cluster.
provider "kubernetes" {
host = module.aks.host
client_certificate = base64decode(module.aks.kube_config.client_certificate)
client_key = base64decode(module.aks.kube_config.client_key)
cluster_ca_certificate = base64decode(module.aks.kube_config.cluster_ca_certificate)
}Test
Configurations
We provide 2 ways to build, run and test the module on a local development machine. Native (Mac/Linux) or Docker.
Native (Mac/Linux)
Prerequisites
Environment setup
We provide simple script to quickly set up module development environment:
$ curl -sSL https://raw.githubusercontent.com/Azure/terramodtest/master/tool/env_setup.sh | sudo bashRun test
Then simply run it in local shell:
$ cd $GOPATH/src/{directory_name}/
$ bundle install
# set service principal
$ export ARM_CLIENT_ID="service-principal-client-id"
$ export ARM_CLIENT_SECRET="service-principal-client-secret"
$ export ARM_SUBSCRIPTION_ID="subscription-id"
$ export ARM_TENANT_ID="tenant-id"
$ export ARM_TEST_LOCATION="eastus"
$ export ARM_TEST_LOCATION_ALT="eastus2"
$ export ARM_TEST_LOCATION_ALT2="westus"
# set aks variables
$ export TF_VAR_client_id="service-principal-client-id"
$ export TF_VAR_client_secret="service-principal-client-secret"
# run test
$ rake build
$ rake fullDocker
We provide a Dockerfile to build a new image based FROM the mcr.microsoft.com/terraform-test Docker hub image which adds additional tools / packages specific for this module (see Custom Image section). Alternatively use only the microsoft/terraform-test Docker hub image by using these instructions.
Prerequisites
Custom Image
This builds the custom image:
$ docker build --build-arg BUILD_ARM_SUBSCRIPTION_ID=$ARM_SUBSCRIPTION_ID --build-arg BUILD_ARM_CLIENT_ID=$ARM_CLIENT_ID --build-arg BUILD_ARM_CLIENT_SECRET=$ARM_CLIENT_SECRET --build-arg BUILD_ARM_TENANT_ID=$ARM_TENANT_ID -t azure-aks .This runs the build and unit tests:
$ docker run --rm azure-aks /bin/bash -c "bundle install && rake build"This runs the end to end tests:
$ docker run --rm azure-aks /bin/bash -c "bundle install && rake e2e"This runs the full tests:
$ docker run --rm azure-aks /bin/bash -c "bundle install && rake full"Requirements
| Name | Version |
|---|---|
| terraform | >= 0.12 |
| azurerm | >= 3.0.0, < 4.0.0 |
| tls | >= 3.3.0, < 4.0.0 |
Providers
| Name | Version |
|---|---|
| azurerm | >= 3.0.0, < 4.0.0 |
| tls | >= 3.3.0, < 4.0.0 |
Modules
No modules.
Resources
| Name | Type |
|---|---|
| azurerm_kubernetes_cluster.main | resource |
| azurerm_log_analytics_solution.main | resource |
| azurerm_log_analytics_workspace.main | resource |
| azurerm_role_assignment.azure_container_registry | resource |
| tls_private_key.main | resource |
| azurerm_resource_group.main | data source |
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| resource_group_name | (Required) The resource group name to be used for the AKS deployment. | string |
n/a | yes |
| admin_username | (Optional) The username of the local administrator to be created on the AKS deployment. | string |
"azureadmin" |
no |
| allowed_maintenance_windows | (Optional) List of allowed Maintenance Windows for AKS. | list(object({ |
[ |
no |
| azure_container_registry_enabled | (Optional) Should the AKS deployment access a Container Registry? | bool |
false |
no |
| azure_container_registry_id | (Optional) The ID of the Container Registry. | string |
null |
no |
| azure_policy_enabled | (Optional) Should the Azure Policy Add-On be enabled? For more details please visit Understand Azure Policy for Azure Kubernetes Service | bool |
false |
no |
| client_id | (Optional) The Client ID (appId) for the Service Principal used for the AKS deployment. | string |
null |
no |
| client_secret | (Optional) The Client Secret (password) for the Service Principal used for the AKS deployment. | string |
null |
no |
| cluster_name | (Optional) The name for the AKS deployment. This variable overwrites the 'prefix' variable. | string |
null |
no |
| default_node_pool_name | (Optional) The name which should be used for the default Kubernetes Node Pool. Changing this forces a new resource to be created. | string |
"default" |
no |
| dns_prefix | (Optional) The DNS prefix for the AKS deployment. This is used to create a unique FQDN for the cluster when it is created. | string |
null |
no |
| dns_service_ip | (Optional) IP address within the Kubernetes service address range that will be used by cluster service discovery (kube-dns). Changing this forces a new resource to be created. | string |
null |
no |
| docker_bridge_cidr | (Optional) IP address (in CIDR notation) used as the Docker bridge IP address on nodes. Changing this forces a new resource to be created. | string |
null |
no |
| enable_auto_scaling | (Optional) Should the Kubernetes Auto Scaler be enabled for this Node Pool? Defaults to false. | bool |
false |
no |
| enable_host_encryption | (Optional) Enable Host Encryption for default Node Pool. Encryption at host feature must be enabled on the subscription: https://docs.microsoft.com/azure/virtual-machines/linux/disks-enable-host-based-encryption-cli | bool |
false |
no |
| enable_node_public_ip | (Optional) Should nodes in this Node Pool have a Public IP Address? Defaults to false. | bool |
false |
no |
| enable_role_based_access_control | (Optional) Enables Role Based Access Control. | bool |
false |
no |
| http_application_routing_enabled | (Optional) Should HTTP Application Routing be enabled? | bool |
false |
no |
| identity_ids | (Optional) Specifies a list of User Assigned Managed Identity IDs to be assigned to this Kubernetes Cluster. This is required when type is set to UserAssigned or SystemAssigned, UserAssigned. | list(string) |
null |
no |
| identity_type | (Optional) Specifies the type of Managed Service Identity that should be configured on this Kubernetes Cluster. Possible values are SystemAssigned, UserAssigned, SystemAssigned, UserAssigned (to enable both). | string |
"SystemAssigned" |
no |
| ingress_application_gateway_enabled | (Optional) Integrates Application Gateway Ingress Controller to this Kubernetes Cluster. Requires an existing Application Gateway. | bool |
false |
no |
| ingress_application_gateway_id | (Optional) The ID of the Application Gateway to integrate as Application Gateway Ingress Controller of this Kubernetes Cluster. | string |
null |
no |
| ingress_application_gateway_name | (Optional) The name of the Application Gateway to be used or created in the Node Pool Resource Group, which in turn will be integrated with the ingress controller of this Kubernetes Cluster. | string |
null |
no |
| ingress_application_gateway_subnet_cidr | (Optional) The subnet CIDR to be used to create an Application Gateway, which in turn will be integrated with the ingress controller of this Kubernetes Cluster. | string |
null |
no |
| ingress_application_gateway_subnet_id | (Optional) The ID of the subnet on which to create an Application Gateway, which in turn will be integrated with the ingress controller of this Kubernetes Cluster. | string |
null |
no |
| key_vault_secrets_provider | (Optional) A key_vault_secrets_provider block. For more details, please visit Azure Keyvault Secrets Provider for AKS. | list(object({ |
[ |
no |
| key_vault_secrets_provider_enabled | (Optional) Enables the Key Vault Secret provider. | bool |
false |
no |
| kubernetes_version | (Optional) Specify which Kubernetes release to use. The default used is the latest Kubernetes version available in the region. | string |
null |
no |
| log_analytics_solution_enabled | (Optional) Enables the Log Analytics Solution for monitoring the Log Analytics Workspace. | bool |
false |
no |
| log_analytics_workspace_enabled | (Optional) Enable the creation of a Log Analytics Workspace. | bool |
true |
no |
| log_analytics_workspace_name | (Optional) If enabled, the name of the Log Analytics Workspace. | string |
null |
no |
| log_analytics_workspace_sku | (Optional) The SKU (pricing level) of the Log Analytics workspace. For new subscriptions the SKU should be set to PerGB2018. | string |
"PerGB2018" |
no |
| log_retention_in_days | (Optional) The retention period in days for logging in the Log Analytics Workspace. | number |
30 |
no |
| max_count | (Optional) The maximum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. | number |
null |
no |
| max_pods | (Optional) The maximum number of pods that can run on each agent. Changing this forces a new resource to be created. | number |
110 |
no |
| min_count | (Optional) The minimum number of nodes which should exist in this Node Pool. If specified this must be between 1 and 1000. | number |
null |
no |
| network_plugin | (Optional) Network plugin to use for networking. Currently supported values are azure and kubenet. Changing this forces a new resource to be created. | string |
"azure" |
no |
| network_policy | (Optional) Sets up network policy to be used with Azure CNI. Network policy allows us to control the traffic flow between pods. Currently supported values are 'calico' and 'azure'. Changing this forces a new resource to be created. | string |
null |
no |
| node_count | (Optional) The number of nodes that should exist in the default Node Pool. This value is ignored when auto-scaling is enabled. | number |
2 |
no |
| node_labels | (Optional) A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. Changing this forces a new resource to be created. | map(string) |
{} |
no |
| node_resource_group | (Optional) The name of the Resource Group where the Kubernetes Nodes should exist. Changing this forces a new resource to be created. | string |
null |
no |
| node_tags | (Optional) A mapping of tags to assign to the Node Pool. | map(string) |
{} |
no |
| node_taints | (Optional) A mapping of taints to assign to the Node Pool. | list(string) |
[] |
no |
| not_allowed_maintenance_windows | (Optional) The start and end of a time span, formatted as an RFC3339 (2022-01-01T00:00:00Z) string. | list(object({ |
[] |
no |
| orchestrator_version | (Optional) Specify which Kubernetes release to use for the orchestration layer. The default used is the latest Kubernetes version available in the region | string |
null |
no |
| os_disk_size_gb | (Optional) The size of the OS Disk which should be used for each agent in the Node Pool. Changing this forces a new resource to be created. | number |
null |
no |
| os_disk_type | (Optional) The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. Changing this forces a new resource to be created. | string |
null |
no |
| os_sku | (Optional) SKU to be used to specify Linux OS. Not applicable to Windows. Possible values include: Ubuntu, CBLMariner. Defaults to Ubuntu. Changing this forces a new resource to be created. | number |
null |
no |
| outbound_type | (Optional) The outbound (egress) routing method which should be used for this Kubernetes Cluster. Possible values are loadBalancer and userDefinedRouting. Defaults to loadBalancer. | string |
"loadBalancer" |
no |
| pod_cidr | (Optional) The CIDR to use for pod IP addresses. This field can only be set when network_plugin is set to kubenet. Changing this forces a new resource to be created. | string |
null |
no |
| prefix | (Optional) The prefix for the resources created in the specified Azure Resource Group. | string |
null |
no |
| private_cluster_enabled | (Optional) Should this Kubernetes Cluster have its API server only exposed on internal IP addresses? This provides a Private IP Address for the Kubernetes API on the Virtual Network where the Kubernetes Cluster is located. Defaults to false. Changing this forces a new resource to be created. | bool |
false |
no |
| private_dns_zone_id | (Optional) Either the ID of Private DNS Zone which should be delegated to this Cluster, System to have AKS manage this or None. In case of None you will need to bring your own DNS server and set up resolving, otherwise cluster will have issues after provisioning. Changing this forces a new resource to be created. | string |
null |
no |
| public_ssh_key | (Optional) A custom SSH key to control access to the AKS deployment. | string |
null |
no |
| rbac_aad_admin_group_object_ids | (Optional) List of Object IDs that are granted admin access. | list(string) |
null |
no |
| rbac_aad_client_app_id | (Optional) The Client ID of an Azure Active Directory Application. | string |
null |
no |
| rbac_aad_managed | (Optional) If set to true, Azure will create/manage a Service Principal used for integration. | bool |
true |
no |
| rbac_aad_server_app_id | (Optional) The Application ID of an Azure Active Directory Application. | string |
null |
no |
| rbac_aad_server_app_secret | (Optional) The Application Secret of an Azure Active Directory Application. | string |
null |
no |
| service_cidr | (Optional) The Network Range used by the Kubernetes service. Changing this forces a new resource to be created. | string |
null |
no |
| service_principal_enabled | (Optional) Should the Azure Policy Add-On be enabled? For more details please visit Understand Azure Policy for Azure Kubernetes Service | bool |
false |
no |
| sku_tier | The SKU Tier that should be used for this Kubernetes Cluster. Possible values are 'Free' and 'Paid'. | string |
"Free" |
no |
| tags | (Optional) A mapping of tags to assign to the resource. | map(string) |
{} |
no |
| type | (Optional) The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets. Defaults to VirtualMachineScaleSets. | string |
"VirtualMachineScaleSets" |
no |
| vm_size | (Optional) The size of the Virtual Machine, such as Standard_DS2_v2. The Microsoft-recommended default size for AKS nodes. | string |
"Standard_DS2_v2" |
no |
| vnet_subnet_id | (Optional) The ID of a Subnet where the Kubernetes Node Pool should exist. Changing this forces a new resource to be created. | string |
null |
no |
| zones | (Optional) Specifies a list of Availability Zones in which this Kubernetes Cluster should be located. Changing this forces a new Kubernetes Cluster to be created. | list(number) |
[ |
no |
Outputs
| Name | Description |
|---|---|
| aks_id | The Kubernetes Managed Cluster ID. |
| http_application_routing_zone_name | The Zone Name of the HTTP Application Routing. |
| identity | The Principal and Tenant IDs associated with this Managed Service Identity. |
| kube_admin_config | Map of credentials to authenticate to Kubernetes as an administrator. |
| kube_admin_config_raw | Raw Kubernetes config for the admin account to be used by kubectl and other compatible tools. This is only available when Role Based Access Control with Azure Active Directory is enabled and local accounts enabled. |
| kube_config | Map of credentials to authenticate to Kubernetes as a user. |
| kube_config_raw | Raw Kubernetes config to be used by kubectl and other compatible tools. |
| kubelet_identity | The Client, Object and User Assigned Identity IDs of the Managed Identity to be assigned to the Kubelets. |
| location | The location where the Managed Kubernetes Cluster is created. |
| log_analytics_workspace_id | The Log Analytics Workspace ID. |
| node_resource_group | The name of the Resource Group where the Kubernetes Nodes should exist. |
| private_key | Private key data in PEM (RFC 1421) and OpenSSH PEM (RFC 4716) format. |
| public_key | Public key data in PEM (RFC 1421) and Authorized Keys format. |
Authors
Originally created by Damien Caro and Malte Lantin
License
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.