/azure-sdwan

Azure Networking patterns for SD-WAN integration

Introduction

This article compliments a two part video series published on YoutTube. Part 1 here covers the strategy angle and "asking the right questions", whilst Part 2 here is the video couterpart to this article, focused on the technical integration patterns for SD-WAN integration in to Azure Virtual Networking

We will jump straight in to the technical scenarios/options. For preamble and context, please watch the videos linked above. NB. You can also click the section headers, to take you straight to video content for that particular topic.

⚠️ Check with your networking vendor for approved/reference designs before continuing

Option 1 - Virtual WAN v1, IPsec API integration

Summary of design option

This design pattern has existed since 2018 when Azure Virtual WAN first went Generally Available. It utilises automation between the networking vendors SD-WAN controller and API's exposed by the Azure Virtual WAN Hub. These APIs offer programmatic automation for attachment of the SD-WAN to Cloud. Example, a new site comes online in Chicago, it is tagged as "cloud enabled" in the SD-WAN vendors software. IPsec tunnels are automatically built from the Branch CPE to Azure Virtual WAN.

Design characteristics

Design area Details
Network Transport IPsec only
Azure Networking category Azure Virtual WAN
Azure/Cloud CPE device Azure Virtual WAN VPN Gateway
Routing Dynamic
Notes 1000 sites, 2Gbps per site, 20Gbps aggregate 1

Links

Option 2 - Customer managed VNet v1, UDR/Static Routing + NVA

Summary of design option

This design pattern has existed for nearly as long as Azure Virtual Networks have. The ability to insert Network Virtual Appliances (NVAs) is a common function leveraged by customers to insert 3rd party Firewalls in the middle of network flows between endpoints inside or to/from the Cloud. We are able to use this same pattern with SD-WAN virtual appliances.

Design characteristics

Design area Details
Network Transport SD-WAN
Azure Networking category Virtual Network (customer managed)
Azure/Cloud CPE device Vendor NVA
Routing Static
Notes Not only will you require static UDR on Spoke Virtual Networks to point SD-WAN bound traffic at your SD-WAN NVA, the SD-WAN appliance itself will have to internally originate static routes that represent your Azure address space. Consider your HA pattern, will you use Azure Load Balancer Active/Active or UDR Rewrite Active/Standby; this will largely be determined by any advanced stateful security features you may be using as part of your SD-WAN deployment.

Links

Option 3 - Customer managed VNet v2, BGP-over-IPsec with VPN Gateway + NVA

Summary of design option

This design pattern looked to enhance #2 with a workaround (trick!) related to Azure VPN Gateways. Historically, VPN Gateways were the only endpoint to which you could configure custom BGP peerings. By establishing IPsec tunnels between the NVA in the Cloud, and the Azure VPN Gateway in the Cloud (now possible using private IPs), we are able to dynamically exchange routes. I.e. the VPN Gateway was purely a bump in the wire, to take BGP information from your NVA, and use it's native fabric access to inject routes in to the Azure SDN.

Design characteristics

Design area Details
Network Transport SD-WAN
Azure Networking category Virtual Network (customer managed)
Azure/Cloud CPE device Vendor NVA
Routing Dynamic
Notes Check vendor supports eBGP multi-hop. Maximum 4 IPsec tunnels, offering approximately 1.25Gbps *4 = 5Gbps throughput aggregate. 2

Option 4 - Virtual WAN v2, "NVA in the Hub"

Summary of design option

This design pattern has existed since mid 2020. In addition to the APIs exposed within Azure Virtual WAN as part of pattern #1, additional, deeper, integrations were opened up for networking partners. In this model, after collaboratively working with Microsoft on the required integration, network vendors are able to place their native virtual appliances directly inside of the Virtual WAN Hub. These appliances then appear as any other Virtual WAN component; that is, entirely managed, with no need to worry about the actual resource, resilience, scaling, feeding and watering etc. In my opinion this offers the deepest levels of Azure integration, it is effectively a PaaS service for SD-WAN endpoint in the Cloud. This integration, also grants the NVA special access to the SDN in order to dynamically exchange routes as if it were a native service.

Design characteristics

Design area Details
Network Transport SD-WAN
Azure Networking category Azure Virtual WAN
Azure/Cloud CPE device Managed (PaaS) vendor NVA
Routing Dynamic
Notes Limited vendor support, see below link

Links

Option 5 - Customer managed VNet v3, Azure Route Server + NVA

Summary of design option

Azure Route Server (ARS) went GA in September 2021. This pattern effectively supersedes pattern #3; now that we can talk BGP directly to ARS, there is no need to deal with the overhead/technical-hassle of running this routing protocol over IPsec. ARS acts as a route reflector, allowing the VNets (local and peered) to learn of your SD-WAN routes, and vice-versa. Note "route reflector", ARS does not sit in the data-path by design.

Design characteristics

Design area Details
Network Transport SD-WAN
Azure Networking category Virtual Network (customer managed)
Azure/Cloud CPE device vendor NVA
Routing Dynamic
Notes Pay attention to ARS limits, approach summarization with care, especially in migration scenarios 3. Check vendor supports eBGP multi-hop.

Links

Option 6 - Virtual WAN v3, BGP Endpoint + NVA

Summary of design option

If you are using Azure Virtual WAN and your favorite networking vendor is not available for pattern #4 then this might be the topology for you. Similar to Azure Route Server, BGP Endpoint for Virtual WAN exposes the ability to directly BGP peer. We are then able to place our NVA in an attached Virtual Network Spoke, and establish BGP Peering sessions with the hub over private IPs (over VNet peering).

Design characteristics

Design area Details
Network Transport SD-WAN
Azure Networking category Azure Virtual WAN
Azure/Cloud CPE device vendor NVA
Routing Dynamic
Notes -

Links

Option 7 - Cloud Edge Colocation, Physical appliance

Summary of design option

An alternative approach. Instead of stretching our SD-WAN solution all the way in to Azure, and in to our Virtual Networks that ride upon the Azure SDN, we may decide to only stretch our SD-WAN network as far as our cloud edge PoPs. This is for those customers who have racks located within collocation centres that are strategically placed in close proximity to major cloud vendors. In this model, the SD-WAN does it's job in getting your business network over the public Internet, and the "last cloud mile" is handled by traditional technologies, normally ExpressRoute. This is a great choice if you have either large (>10Gbps) throughput requirements, or extremely complicated multi-vrf networks, wherein the physical SD-WAN appliances can be sized and specified appropriately.

Design characteristics

Design area Details
Network Transport SD-WAN to Cloud Edge, ExpressRoute to Azure
Azure Networking category n/a
Azure/Cloud CPE device n/a
Routing Dynamic
Notes You can solve for the "cloud edge" box in a number of ways; a) Build it yourself, common with large Enterprise, b) Take a Virtual Edge from a company like Equinix or c) Take a Cloud Onramp service as part of your SD-WAN vendor offering. See links below.

Links

Conclusion and options flow-chart

The number of technical options may feel confusing or overwhelming, but I would keep things simple;

  • If you are using customer-managed traditional hub/spoke networking, aim for pattern #5 based on Azure Route Server
  • If you are using Azure Virtual WAN based networking, aim for pattern #4 "NVA in hub", otherwise pattern #6 with "BGP Endpoint".

The non tl;dr version of this is represented in the flow-chart below.

Footnotes

  1. https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#virtual-wan-limits ↩

  2. https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways#benchmark ↩

  3. Whilst a maximum of 1000 routes per NVA BGP peer can be advertised to Azure Route Server, note that if using the branch2branch feature of ARS, this limit effectively drops to 200 total routes across all BGP peers https://docs.microsoft.com/en-us/azure/route-server/route-server-faq#route-server-limits ↩