New-HNSNetwork changes the MTU size from 1500 to 1450 in AKS Windows VMs
AbelHu opened this issue · 1 comments
AbelHu commented
It seems like that someone has observed it in https://raw.githubusercontent.com/pjh/SDN/mtu-with-new-hnsnetwork/mtu-with-new-hnsnetwork.txt.
#####################################################################
## The just-created L2Bridge type HNS network subsumes the previous
## Ethernet interface with the "vEthernet (Ethernet) 2" interface
## which for some reason has an MTU of 1450 rather than 1432 or 1500.
## Also, the MTU of the existing "vEthernet (nat)" interface has been
## decreased by 50 bytes!
#####################################################################
Reference: TCP/IP performance tuning for Azure VMs | Microsoft Docs
Azure and VM MTU
The default MTU for Azure VMs is 1,500 bytes. The Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes.
Note that the Virtual Network stack isn't inherently inefficient because it fragments packets at 1,400 bytes even though VMs have an MTU of 1,500. A large percentage of network packets are much smaller than 1,400 or 1,500 bytes.
Below is the command in provisioning AKS Windows nodes.
$externalNetwork = "ext"
$nas = @(Get-NetAdapter -Physical)
foreach($na in $nas)
{
$netIP = Get-NetIPAddress -ifIndex $na.ifIndex -AddressFamily IPv4 -ErrorAction SilentlyContinue -ErrorVariable netIPErr
if ($netIP)
{
$managementIP = $netIP.IPAddress
$adapterName = $na.Name
break
}
else {
Write-Error "No IPv4 found on the network adapter $($na.Name); trying the next adapter ..."
if ($netIPErr) {
Write-Host "error when retrieving IPAddress: $netIPErr"
$netIPErr.Clear()
}
}
}
if(-Not $managementIP)
{
throw "None of the physical network adapters has an IP address"
}
Write-Host "Using adapter $adapterName with IP address $managementIP"
New-HNSNetwork -Type "L2Bridge" -AddressPrefix "192.168.255.0/30" -Gateway "192.168.255.1" -AdapterName $adapterName -Name $externalNetwork -Verbose
Logs:
Getting MTU size
MTU MediaSenseState Bytes In Bytes Out Interface
------ --------------- --------- --------- -------------
4294967295 1 0 0 Loopback Pseudo-Interface 1
1500 1 0 1988 vEthernet (nat)
1500 1 35399353 1419013 Ethernet 2
2022-03-24T04:46:56.1914010+00:00: Creating new HNS network "ext"
2022-03-24T04:46:59.6000519+00:00: No IPv4 found on the network adapter Ethernet 3; trying the next adapter ...
2022-03-24T04:46:59.6000519+00:00: error when retrieving IPAddress: No matching MSFT_NetIPAddress objects found by CIM query for instances of the ROOT/StandardCimv2/MSFT_NetIPAddress class on the CIM server: SELECT * FROM MSFT_NetIPAddress WHERE ((InterfaceIndex = 17)) AND ((AddressFamily = 2)). Verify query parameters and retry.
2022-03-24T04:46:59.6156744+00:00: Using adapter Ethernet 2 with IP address 10.240.1.151
VERBOSE: Invoke-HNSRequest Method[POST] Path[/networks] Data[{
"Subnets": [
{
"GatewayAddress": "192.168.255.1",
"AddressPrefix": "192.168.255.0/30"
}
],
"NetworkAdapterName": "Ethernet 2",
"Name": "ext",
"Type": "L2Bridge"
}]
VERBOSE: Result :
{"Output":{"ActivityId":"F8905258-1CFD-4B88-95F8-072C9B663F39","AdditionalParams":{},"CurrentEndpointCount":0,"Extensio
ns":[{"Id":"E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A","IsEnabled":false,"Name":"Microsoft Windows Filtering
Platform"},{"Id":"E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017","IsEnabled":true,"Name":"Microsoft Azure VFP Switch
Extension"},{"Id":"EA24CD6C-D17A-4348-9190-09F0D5BE83DD","IsEnabled":true,"Name":"Microsoft NDIS
Capture"}],"Flags":0,"Health":{"LastErrorCode":0,"LastUpdateTime":132925708223917418},"ID":"7F4003A8-5E19-4414-990F-72B
19796E78E","IPv6":false,"LayeredOn":"798783E7-1A28-4CDD-9CDC-F438A4E30B46","MacPools":[{"EndMacAddress":"00-15-5D-9D-DF
-FF","StartMacAddress":"00-15-5D-9D-D0-00"}],"ManagementIP":"10.240.1.151","MaxConcurrentEndpoints":0,"Name":"ext","Net
workAdapterName":"Ethernet
2","Policies":[],"Resources":{"AdditionalParams":{},"AllocationOrder":0,"Health":{"LastErrorCode":0,"LastUpdateTime":13
2925708267108672},"ID":"F8905258-1CFD-4B88-95F8-072C9B663F39","PortOperationTime":0,"State":1,"SwitchOperationTime":0,"
VfpOperationTime":0,"parentId":"6C5D500F-C4C8-45CA-8BBB-730C5E13D338"},"State":1,"Subnets":[{"AdditionalParams":{},"Add
ressPrefix":"192.168.255.0/30","GatewayAddress":"192.168.255.1","Health":{"LastErrorCode":0,"LastUpdateTime":1329257082
23917418},"ID":"76E15ADC-3AE9-4DF7-90C4-5FF342E88901","ObjectType":5,"Policies":[],"State":0}],"TotalEndpoints":0,"Type
":"L2Bridge","Version":38654705669},"Success":true}
ActivityId : F8905258-1CFD-4B88-95F8-072C9B663F39
AdditionalParams :
CurrentEndpointCount : 0
Extensions : {@{Id=E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A; IsEnabled=False; Name=Microsoft Windows Filtering
Platform}, @{Id=E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017; IsEnabled=True; Name=Microsoft Azure
VFP Switch Extension}, @{Id=EA24CD6C-D17A-4348-9190-09F0D5BE83DD; IsEnabled=True;
Name=Microsoft NDIS Capture}}
Flags : 0
Health : @{LastErrorCode=0; LastUpdateTime=132925708223917418}
ID : 7F4003A8-5E19-4414-990F-72B19796E78E
IPv6 : False
LayeredOn : 798783E7-1A28-4CDD-9CDC-F438A4E30B46
MacPools : {@{EndMacAddress=00-15-5D-9D-DF-FF; StartMacAddress=00-15-5D-9D-D0-00}}
ManagementIP : 10.240.1.151
MaxConcurrentEndpoints : 0
Name : ext
NetworkAdapterName : Ethernet 2
Policies : {}
Resources : @{AdditionalParams=; AllocationOrder=0; Health=; ID=F8905258-1CFD-4B88-95F8-072C9B663F39;
PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
parentId=6C5D500F-C4C8-45CA-8BBB-730C5E13D338}
State : 1
Subnets : {@{AdditionalParams=; AddressPrefix=192.168.255.0/30; GatewayAddress=192.168.255.1; Health=;
ID=76E15ADC-3AE9-4DF7-90C4-5FF342E88901; ObjectType=5; Policies=System.Object[]; State=0}}
TotalEndpoints : 0
Type : L2Bridge
Version : 38654705669
2022-03-24T04:47:06.7421215+00:00: It took 7 seconds to create the ext network.
Getting MTU size
MTU MediaSenseState Bytes In Bytes Out Interface
------ --------------- --------- --------- -------------
4294967295 1 0 0 Loopback Pseudo-Interface 1
1450 1 0 1988 vEthernet (nat)
1450 1 2676 3066 vEthernet (Ethernet 2)
AbelHu commented
We have closed the issue in AKS with "It is by design that HNS sets the MTU from 1500 to 1450 for process-isolated containers". It should be better if HNS can share more details for this design. Thanks.