terraform-azurerm-avm-ptn-aks-production

NOTE: This module follows the semantic versioning and versions prior to 1.0.0 should be consider pre-release versions.

This is the Production Standard for AKS pattern module for Azure Verified Modules (AVM) library. This module deploys a production standard AKS cluster along with supporting a Virtual Network and Azure container registry. It provisions an environment sufficient for most production deployments for AKS. It leverages the AzureRM provider and sets a number of initial defaults to minimize the overall inputs for simple configurations.

Major version Zero (0.y.z) is for initial development. Anything MAY change at any time. A module SHOULD NOT be considered stable till at least it is major version one (1.0.0) or greater. Changes will always be via new versions being published and no changes will be made to existing published versions. For more details please go to https://semver.org/

Deployment Steps

Set up a GitHub repo environment called test.
Configure environment protection rule to ensure that approval is required before deploying to this environment.
Create a user-assigned managed identity in your test subscription.
Create a role assignment for the managed identity on your test subscription, use the minimum required role.
Configure federated identity credentials on the user assigned managed identity. Use the GitHub environment.
Create the following environment secrets on the test environment:
1. AZURE_CLIENT_ID
2. AZURE_TENANT_ID
3. AZURE_SUBSCRIPTION_ID
Search and update TODOs within the code and remove the TODO comments once complete.

Requirements

The following requirements are needed by this module:

terraform (>= 1.3.0)
azapi (>= 1.4.0, < 2.0)
azurerm (>= 3.86.0)
local (2.4.1)
null (>= 3.0)
random (>= 3.5.0)

Providers

The following providers are used by this module:

azapi (>= 1.4.0, < 2.0)
azurerm (>= 3.86.0)
local (2.4.1)
null (>= 3.0)
random (>= 3.5.0)

Resources

The following resources are used by this module:

azapi_update_resource.aks_cluster_post_create (resource)
azurerm_container_registry.this (resource)
azurerm_kubernetes_cluster.this (resource)
azurerm_kubernetes_cluster_node_pool.this (resource)
azurerm_log_analytics_workspace.this (resource)
azurerm_log_analytics_workspace_table.this (resource)
azurerm_management_lock.this (resource)
azurerm_monitor_diagnostic_setting.aks (resource)
azurerm_private_endpoint.this (resource)
azurerm_private_endpoint_application_security_group_association.this (resource)
azurerm_resource_group_template_deployment.telemetry (resource)
azurerm_role_assignment.acr (resource)
azurerm_role_assignment.this (resource)
azurerm_user_assigned_identity.aks (resource)
null_resource.kubernetes_version_keeper (resource)
random_id.telem (resource)
random_string.acr_suffix (resource)
local_file.compute_provider (data source)
local_file.locations (data source)

Required Inputs

The following input variables are required:

location

Description: The Azure region where the resources should be deployed.

Type: string

name

Description: The name for the AKS resources created in the specified Azure Resource Group. This variable overwrites the 'prefix' var (The 'prefix' var will still be applied to the dns_prefix if it is set)

Type: string

resource_group_name

Description: The resource group where the resources will be deployed.

Type: string

Optional Inputs

The following input variables are optional (have default values):

agents_tags

Description: (Optional) A mapping of tags to assign to the Node Pool.

Type: map(string)

Default: {}

client_id

Description: (Optional) The Client ID (appId) for the Service Principal used for the AKS deployment

Type: string

Default: ""

client_secret

Description: (Optional) The Client Secret (password) for the Service Principal used for the AKS deployment

Type: string

Default: ""

customer_managed_key

Description: Customer managed keys that should be associated with the resource.

Type:

object({
    key_vault_resource_id              = optional(string)
    key_name                           = optional(string)
    key_version                        = optional(string, null)
    user_assigned_identity_resource_id = optional(string, null)
  })

Default: {}

enable_telemetry

Description: This variable controls whether or not telemetry is enabled for the module.
For more information see https://aka.ms/avm/telemetryinfo.
If it is set to false, then no telemetry will be collected.

Type: bool

Default: true

identity_ids

Description: (Optional) Specifies a list of User Assigned Managed Identity IDs to be assigned to this Kubernetes Cluster.

Type: list(string)

Default: null

key_vault_secrets_provider_enabled

Description: (Optional) Whether to use the Azure Key Vault Provider for Secrets Store CSI Driver in an AKS cluster. For more details: https://docs.microsoft.com/en-us/azure/aks/csi-secrets-store-driver

Type: bool

Default: false

kubernetes_version

Description: Specify which Kubernetes release to use. Specify only minor version, such as '1.28'.

Type: string

Default: null

lock

Description: The lock level to apply. Default is None. Possible values are None, CanNotDelete, and ReadOnly.

Type:

object({
    name = optional(string, null)
    kind = optional(string, "None")
  })

Default: {}

log_analytics_workspace_id

Description: (Optional) The ID of the Log Analytics Workspace to use for the OMS agent.

Type: string

Default: null

managed_identities

Description: Managed identities to be created for the resource.

Type:

object({
    system_assigned            = optional(bool, false)
    user_assigned_resource_ids = optional(set(string), [])
  })

Default: {}

monitor_metrics

Description: (Optional) Specifies a Prometheus add-on profile for the Kubernetes Cluster
object({
annotations_allowed = "(Optional) Specifies a comma-separated list of Kubernetes annotation keys that will be used in the resource's labels metric."
labels_allowed = "(Optional) Specifies a Comma-separated list of additional Kubernetes label keys that will be used in the resource's labels metric." })

Type:

object({
    annotations_allowed = optional(string)
    labels_allowed      = optional(string)
  })

Default: null

node_cidr

Description: (Optional) The CIDR to use for node IPs in the Kubernetes cluster. Changing this forces a new resource to be created.

Type: string

Default: null

node_pools

Description: A map of node pools that need to be created and attached on the Kubernetes cluster. The key of the map can be the name of the node pool, and the key must be static string. The value of the map is a node_pool block as defined below:
map(object({
name = (Required) The name of the Node Pool which should be created within the Kubernetes Cluster. Changing this forces a new resource to be created. A Windows Node Pool cannot have a name longer than 6 characters. A random suffix of 4 characters is always added to the name to avoid clashes during recreates.
vm_size = (Required) The SKU which should be used for the Virtual Machines used in this Node Pool. Changing this forces a new resource to be created.
orchestrator_version = (Required) The version of Kubernetes which should be used for this Node Pool. Changing this forces a new resource to be created.
max_count = (Optional) The maximum number of nodes which should exist within this Node Pool. Valid values are between 0 and 1000 and must be greater than or equal to min_count.
min_count = (Optional) The minimum number of nodes which should exist within this Node Pool. Valid values are between 0 and 1000 and must be less than or equal to max_count.
os_sku = (Optional) Specifies the OS SKU used by the agent pool. Possible values include: Ubuntu, CBLMariner, Mariner, Windows2019, Windows2022. If not specified, the default is Ubuntu if OSType=Linux or Windows2019 if OSType=Windows. And the default Windows OSSKU will be changed to Windows2022 after Windows2019 is deprecated. Changing this forces a new resource to be created.
mode = (Optional) Should this Node Pool be used for System or User resources? Possible values are System and User. Defaults to User.
os_disk_size_gb = (Optional) The Agent Operating System disk size in GB. Changing this forces a new resource to be created.
tags = (Optional) A mapping of tags to assign to the resource. At this time there's a bug in the AKS API where Tags for a Node Pool are not stored in the correct case - you may wish to use Terraform's ignore_changes functionality to ignore changes to the casing until this is fixed in the AKS API.
zones = (Optional) Specifies a list of Availability Zones in which this Kubernetes Cluster Node Pool should be located. Changing this forces a new Kubernetes Cluster Node Pool to be created. }))

Type:

map(object({
    name                 = string
    vm_size              = string
    orchestrator_version = string
    # do not add nodecount because we enforce the use of auto-scaling
    max_count       = optional(number)
    min_count       = optional(number)
    os_sku          = optional(string)
    mode            = optional(string)
    os_disk_size_gb = optional(number, null)
    tags            = optional(map(string), {})
    zones           = optional(set(string))
  }))

Default: {}

orchestrator_version

Description: Specify which Kubernetes release to use. Specify only minor version, such as '1.28'.

Type: string

Default: null

pod_cidr

Description: (Optional) The CIDR to use for pod IPs in the Kubernetes cluster. Changing this forces a new resource to be created.

Type: string

Default: null

private_endpoints

Description: A map of private endpoints to create on this resource. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.

name - (Optional) The name of the private endpoint. One will be generated if not set.
role_assignments - (Optional) A map of role assignments to create on the private endpoint. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time. See var.role_assignments for more information.
lock - (Optional) The lock level to apply to the private endpoint. Default is None. Possible values are None, CanNotDelete, and ReadOnly.
tags - (Optional) A mapping of tags to assign to the private endpoint.
subnet_resource_id - The resource ID of the subnet to deploy the private endpoint in.
private_dns_zone_group_name - (Optional) The name of the private DNS zone group. One will be generated if not set.
private_dns_zone_resource_ids - (Optional) A set of resource IDs of private DNS zones to associate with the private endpoint. If not set, no zone groups will be created and the private endpoint will not be associated with any private DNS zones. DNS records must be managed external to this module.
application_security_group_resource_ids - (Optional) A map of resource IDs of application security groups to associate with the private endpoint. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.
private_service_connection_name - (Optional) The name of the private service connection. One will be generated if not set.
network_interface_name - (Optional) The name of the network interface. One will be generated if not set.
location - (Optional) The Azure location where the resources will be deployed. Defaults to the location of the resource group.
resource_group_name - (Optional) The resource group where the resources will be deployed. Defaults to the resource group of this resource.
ip_configurations - (Optional) A map of IP configurations to create on the private endpoint. If not specified the platform will create one. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.
- name - The name of the IP configuration.
- private_ip_address - The private IP address of the IP configuration.

Type:

map(object({
    name = optional(string, null)
    role_assignments = optional(map(object({
      role_definition_id_or_name             = string
      principal_id                           = string
      description                            = optional(string, null)
      skip_service_principal_aad_check       = optional(bool, false)
      condition                              = optional(string, null)
      condition_version                      = optional(string, null)
      delegated_managed_identity_resource_id = optional(string, null)
    })), {})
    lock = optional(object({
      name = optional(string, null)
      kind = optional(string, "None")
    }), {})
    tags                                    = optional(map(any), null)
    subnet_resource_id                      = string
    private_dns_zone_group_name             = optional(string, "default")
    private_dns_zone_resource_ids           = optional(set(string), [])
    application_security_group_associations = optional(map(string), {})
    private_service_connection_name         = optional(string, null)
    network_interface_name                  = optional(string, null)
    location                                = optional(string, null)
    resource_group_name                     = optional(string, null)
    ip_configurations = optional(map(object({
      name               = string
      private_ip_address = string
    })), {})
  }))

Default: {}

rbac_aad_admin_group_object_ids

Description: Object ID of groups with admin access.

Type: list(string)

Default: null

rbac_aad_azure_rbac_enabled

Description: (Optional) Is Role Based Access Control based on Azure AD enabled?

Type: bool

Default: null

rbac_aad_tenant_id

Description: (Optional) The Tenant ID used for Azure Active Directory Application. If this isn't specified the Tenant ID of the current Subscription is used.

Type: string

Default: null

role_assignments

Description: A map of role assignments to create on this resource. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.

role_definition_id_or_name - The ID or name of the role definition to assign to the principal.
principal_id - The ID of the principal to assign the role to.
description - The description of the role assignment.
skip_service_principal_aad_check - If set to true, skips the Azure Active Directory check for the service principal in the tenant. Defaults to false.
condition - The condition which will be used to scope the role assignment.
condition_version - The version of the condition syntax. Valid values are '2.0'.

Note: only set skip_service_principal_aad_check to true if you are assigning a role to a service principal.

Type:

map(object({
    role_definition_id_or_name             = string
    principal_id                           = string
    description                            = optional(string, null)
    skip_service_principal_aad_check       = optional(bool, false)
    condition                              = optional(string, null)
    condition_version                      = optional(string, null)
    delegated_managed_identity_resource_id = optional(string, null)
  }))

Default: {}

Outputs

The following outputs are exported:

private_endpoints

Description: A map of private endpoints. The map key is the supplied input to var.private_endpoints. The map value is the entire azurerm_private_endpoint resource.

resource

Description: This is the full output for the resource.

Modules

The following Modules are called:

regions

Source: Azure/regions/azurerm

Version: >= 0.3.0

vnet

Source: Azure/subnets/azurerm

Version: 1.0.0

Data Collection

The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.

mosabami/terraform-azurerm-avm-ptn-aks-production

terraform-azurerm-avm-ptn-aks-production

NOTE: This module follows the semantic versioning and versions prior to 1.0.0 should be consider pre-release versions.

Deployment Steps

Requirements

Providers

Resources

Required Inputs

Optional Inputs

Outputs

Modules

Data Collection