terraform-azurerm-alz-subnet

Description

This module is used to deploy subnet with network security group (NSG) and route table (RT) associated as workaround to the azurerm_subnet resource so it doesn't conflict with the Azure Landing Zone policies Subnets should have a Network Security Group and Subnets should have a User Defined Route which are commonly used within large/medium sized enterprises in the Azure Landing Zone Reference Architecture.

Why

Why not just use the terraform resource azurerm_subnet to deploy the subnet within the virtual network deployed by the e.g Azure Landing Zone vending module ?

That is because it's known for having a bug that conflicts with these deny policies due to how the resouce is designed, and how it's not able to associate the Network Security Group (NSG) and the Route Table (RT) to the subnet when the resouce is deployed, due to those restrictions in the resource it needs to use seperate resources for that action azurerm_subnet_network_security_group_association and the azurerm_route_table_association but unfortunately that conflicts with the Deny policies and doesn't allow you to create the subnet.

The only way to deploy a subnet that doesn't conflict with these policies is to use the azurerm_virtual_network resource to deploy the subnet and reference the network security group, and the route table in one API call.

However, that only resolves the part of the problem as the azurerm_virtual_network resouce doesn't support complex configurations on the subnet like Subnet Delegation, Service Endpoints and Private link-/ endpoint network policies

In addition to prevent autonomous work teams/owners/contributors of their Landing Zone to create a subnet within their own code repository, and constant need to involve the Platform Team to create the subnets for the landing zones.

Issues that have been previously been posted and resolved with workarounds on this bug when creating subnet while adhearing to these policies.

These github issues have been around since mid 2019, some of them have been resolved with workarounds, workaround similar to what is being used in this module, and some are still open.

Known Issues in this module.

Please have a look at the Known Issues section for more information, before posting an issue on this repository.

Requirements

Name	Version
terraform	>= 1.3
azapi	>= 1.9, < 2.0
azurerm	>= 3.11, < 4.0

Simple module usage

# Simple usage to showcase the functionality of the module

########################
# Pre-requisites Setup #
########################

resource "azurerm_resource_group" "this" {
  name     = "rg-test-01"
  location = "westeurope"
}

resource "azurerm_virtual_network" "this" {
  name                = "vnet-alz-01"
  address_space       = ["10.30.0.0/24"]
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

# Pre-creating route table to showcase simple usage of the module
resource "azurerm_route_table" "this" {
  name                = "rt-alz-01"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

# Pre-creating network security group to showcase simple usage of the module
resource "azurerm_network_security_group" "this" {
  name                = "nsg-alz-01"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

########################
#    Example Usage     #
########################

module "subnet" {
  source                        = "haflidif/alz-subnet/azurerm"
  subnet_name                   = "snet-alz-01"
  address_prefixes              = ["10.30.0.0/27"]
  virtual_network_resource_id   = azurerm_virtual_network.this.id
  location                      = azurerm_resource_group.this.location
  create_network_security_group = false
  create_route_table            = false
  route_table_id                = azurerm_route_table.this.id
  network_security_group_id     = azurerm_network_security_group.this.id
}

See more usage examples here:

ℹ️ Note:
Otherwise, see the full module test here: ▶️ test/autotest

Resources

Name	Type
azapi_resource.subnet	resource
azurerm_network_security_group.nsg	resource
azurerm_resource_group.sub_resouces_rg	resource
azurerm_route.default_route_to_nva	resource
azurerm_route_table.route_table	resource

Inputs

Name	Description	Type	Default	Required
address_prefixes	(Required) The Address prefix to use for the subnet.	`list(string)`	n/a	yes
location	(Required) Specifies the Azure location where the resources should be created. Changing this forces a new resource to be created.	`string`	n/a	yes
subnet_name	(Required) Name of the subnet. Changing this forces a new resource to be created.	`string`	n/a	yes
virtual_network_resource_id	(Required) The ID of the virtual network where the subnet should be created. Changing this forces a new resource to be created.	`string`	n/a	yes
create_network_security_group	(Optional) Boolean flag which controls if network security group should be created. Defaults to `true`. Set to `false` and provide value for `network_security_group_id` to reference existing network security group.	`bool`	`true`	no
create_route_table	(Optional) Boolean flag which controls if route table should be created. Defaults to `true`. Set to `false` and provide value for `route_table_id` to reference existing route table.	`bool`	`true`	no
delegation_service_name	(Optional) Provide the service name for the subnet delegation configuration.	`string`	`""`	no
disable_bgp_route_propagation	(Optional) Boolean flag which controls propagation of routes learned by BGP on that route table. `true` means disable. Defaults to `false`, when used in combination with `create_sub_network_resources` and 'nva_ip_address' this should be set to `true` to prevent the default 0.0.0.0/0 route to be propagated in the route table via BGP.	`bool`	`false`	no
network_security_group_id	(Optional) The ID of existing network security group to associate with the subnet.	`string`	`""`	no
network_security_group_name	(Optional) The name of the new network security group to associate with the subnet.	`string`	`""`	no
nva_ip_address	(Optional) The IP address of the network virtual appliance often a firewall, located in a hub virtual network, this is used to create user defined route to route 0.0.0.0/0 traffic to the network virtual appliance.	`string`	`""`	no
private_endpoint_network_policies_enabled	(Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to `true` will Enable the policy and setting this to `false` will Disable the policy. Defaults to `true`	`bool`	`true`	no
private_link_service_network_policies_enabled	(Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to `true` will Enable the policy and setting this to `false` will Disable the policy. Defaults to `true`	`bool`	`true`	no
route_table_id	(Optional) The ID of existing route table to associate with the subnet.	`string`	`""`	no
route_table_name	(Optional) The name of the new route table to associate with the subnet.	`string`	`""`	no
service_endpoint_names	(Optional) List of service endpoints to associate with the subnet.	`list(string)`	`[]`	no
sub_resource_group_name	(Optional) The name of the resource group where the sub-resources will be created. Changing this forces a new resource to be created.	`string`	`""`	no
tags	(Optional) A mapping of tags to assign to the resource.	`map(string)`	`{}`	no

Outputs

Name	Description
subnet_id	Virtual Network Subnet resource id
subnet_nsg_id	Network Security Group resource id
subnet_route_table_id	Route Table resource id

Modules

No modules.

Argument Reference

The following arguments are supported:

subnet_name - (Required) Name of the subnet. Changing this forces a new resource to be created.
address_prefixes - (Required) The Address prefix to use for the subnet.

ℹ️ NOTE:
Currently only a single address prefix can be set as the Multiple Subnet Address Prefixes Feature is not yet in public preview or general availability.
virtual_network_resource_id - (Required) The ID of the virtual network where the subnet should be created. Changing this forces a new resource to be created.
location - (Required) Specifies the Azure location where the resources should be created. Changing this forces a new resource to be created.
create_network_security_group - (Optional) Boolean flag which controls if network security group should be created. Defaults to true. Set to false and provide value for network_security_group_id to reference existing network security group.

ℹ️ NOTE:
If this is set to false the following argument is required network_security_group_id if it is not set then the deployment will conflict with the Azure Policies and subnet can't be deployed in compliance with the policy.
network_security_group_id - (Optional) The ID of existing network security group to associate with the subnet, make sure the create_sub_network_resources is set to false if you want to reference an existing network security group.
network_security_group_name - (Optional) The name of the new network security group to associate with the subnet.
create_route_table - (Optional) Boolean flag which controls if route table should be created. Defaults to true. Set to false and provide value for route_table_id to reference existing route table.

ℹ️ NOTE:
If this is set to false the following argument is required route_table_id if it is not set then the deployment will conflict with the Azure Policies and subnet can't be deployed in compliance with the policy.
route_table_id - (Optional) The ID of existing route table to associate with the subnet.
route_table_name - (Optional) The name of the new route table to associate with the subnet.
nva_ip_address - (Optional) The IP address of the network virtual appliance often a firewall, located in a hub virtual network, this is used to create user defined route to route

ℹ️ NOTE:
Specify the IP address of the network virtual appliance often a firewall, located in a hub virtual network, this is used to create user defined route in the new route table to route 0.0.0.0/0 to the network virtual appliance.
sub_resource_group_name - (Optional) The name of the resource group where the sub-resources will be created. Changing this forces a new resource to be created.
delegation_service_name - (Optional) Provide the service name for the subnet delegation configuration.

ℹ️ NOTE:
Delegating to service may not be available in all regions. Check if the service you are delegating to is available in your region using the Azure CLI.
service_endpoint_names - (Optional) List of service endpoints to associate with the subnet, multiple service endpoints are supported.

ℹ️ NOTE:
In short Service Endpoints are used to secure Azure service resources to use the virtual network instead of the public internet, utilizing the azure backbone network. Multiple Service Endpoints can be defined on each subnet.

Generally available service endpoints for all regions are:
Microsoft.Storage, Microsoft.Storage.Global, Microsoft.Sql, Microsoft.AzureCosmosDB, Microsoft.KeyVault, Microsoft.ServiceBus, Microsoft.EventHub, Microsoft.AzureActiveDirectory, Microsoft.Web, Microsoft.CognitiveServices

Public Preview:
Microsoft.ContainerRegistry

For the most up-to-date notification and list of available service endpoints in your region check the Azure virtual network service endpoints documentation.
private_link_service_network_policies_enabled - (Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to true will Enable the policy and setting this to false will Disable the policy. Defaults to true

ℹ️ NOTE:
Private Link Service Network Policies are enabled by default to ensure that traffic from Private Link services go through the Network Security Group and uses the User Defined Routes in the route table associated with the subnet.
If this is set to false the traffic for all private link services will bypass the Network Security Group and the User Defined Routes in the route table associated with the subnet.
private_endpoint_network_policies_enabled - (Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to true will Enable the policy and setting this to false will Disable the policy. Defaults to true

ℹ️ NOTE:
Private Endpoint Network Policies are enabled by default to ensure that traffic from the private endpoint go through the Network Security Group and uses the User Defined Routes in the route table associated with the subnet.
If this is set to false the traffic for all private endpoints in the subnet will bypass the Network Security Group and the User Defined Routes in the route table associated with the subnet.
disable_bgp_route_propagation - (Optional) Boolean flag which controls propagation of routes learned by BGP on that route table. true means disable. Defaults to false, when used in combination with create_sub_network_resources and nva_ip_address this should be set to true to override the default route 0.0.0.0/0 and to prevent routes learned by BGP (Route Propagation) to bypass the network virtual appliance.
tags - (Optional) A mapping of tags to assign to the resource.

Authors

Originally created by Haflidi Fridthjofsson

Other Resources

Known Issues

Response 409: 409 Conflict, Error Code: AnotherOperationInProgress

The Azure API might throw an Response 409: 409 Conflict, Error Code: AnotherOperationInProgress when creating the subnet, this is due to dely in the Azure API when the virtual network resource is being updated and another create/update/delete is ran at the same time, in parallel or before the Azure API catches up with the previous operation.

| --------------------------------------------------------------------------------
│ RESPONSE 409: 409 Conflict
│ ERROR CODE: AnotherOperationInProgress
│ --------------------------------------------------------------------------------
│ {
│   "error": {
│     "code": "AnotherOperationInProgress",
│     "message": "Another operation on this or dependent resource is in progress. To retrieve status of the operation use uri: https://management.azure.com/subscriptions/<GUID_REMOVED>/providers/Microsoft.Network/locations/westeurope/operations/<GUID_REMOVED>?api-version=2023-04-01.",
│     "details": []
│   }
│ }
│ --------------------------------------------------------------------------------

This happens particularly when calling the module multiple times in the same terraform run, the workaround is to use explicit depends_on on the module resource to ensure that the module is ran sequentially e.g.

module "subnet1" {
  source = "haflidif/alz-subnet/azurerm"
  ...ommitted for brevity
}

module "subnet2" {
  source = "haflidif/alz-subnet/azurerm"
  ...ommitted for brevity
  depends_on = [ module.subnet1 ]
}

See the following testing mechanism being used to test the module with multiple subnets in the same terraform run:
▶️ test/autotest

Please submit a issue on this repository if you find a better workaround or solution to this issue.

ℹ️ NOTE:
It's a known issue not specifically related to this module, but to the azurerm provider and the Azure API and is being tracked in the following github issue: ▶️ Subnets on same vnet fail due to parrallel setup #3780

dummy-andra/terraform-azurerm-alz-subnet