SDNExpress fails in Stage 5 Adding network interfaces to network controllers
AliveDevil opened this issue · 9 comments
Did someone figure out how to get SDNExpress deploying without failures?
With a bunch of changes already done (as in https://github.com/alivedevil/sdn) to ensure that at least the virtual machine deployment works out, I am buffled that even the default MultiNodeSampleConfig-file fails to configure Stage 5 correctly.
Following SDNExpress log is produced:
AUSFÜHRLICH: [20210402-17:59:23] -HostName: belsa
AUSFÜHRLICH: [20210402-17:59:23] -ComputerName: sdnGW01
AUSFÜHRLICH: [20210402-17:59:23] -RestName: sdn.home.arpa
AUSFÜHRLICH: [20210402-17:59:23] -FrontEndLogicalNetworkName: HNVPA
AUSFÜHRLICH: [20210402-17:59:23] -FrontEndIp: 10.90.90.6
AUSFÜHRLICH: [20210402-17:59:23] -PoolName: DefaultAll
AUSFÜHRLICH: [20210402-17:59:23] -BackEndMac: 001DD8220009
AUSFÜHRLICH: [20210402-17:59:23] -FrontEndMac: 001DD8220008
AUSFÜHRLICH: [20210402-17:59:23] -FrontEndAddressPrefix: 10.90.90.0/24
AUSFÜHRLICH: [20210402-17:59:23] -LocalASN: 65534
AUSFÜHRLICH: [20210402-17:59:23] -Routers: System.Collections.Hashtable
AUSFÜHRLICH: [20210402-17:59:23] -Credential: System.Management.Automation.PSCredential
AUSFÜHRLICH: [20210402-17:59:23] Unbound Arguments:
AUSFÜHRLICH: [20210402-17:59:23] ParameterSet: MultiPeer
AUSFÜHRLICH: [20210402-17:59:23] Begin Invoke-Command output:
AUSFÜHRLICH: [20210402-17:59:24] Finished Invoke-Command output.
AUSFÜHRLICH: [20210402-17:59:24] Begin Invoke-Command output:
AUSFÜHRLICH: [20210402-17:59:25] [sdnGW01] Renaming Network Adapters
AUSFÜHRLICH: [20210402-17:59:28] [sdnGW01] Remote Access multi-tenancy already enabled.
AUSFÜHRLICH: [20210402-17:59:29] [sdnGW01] Enabling gateway service.
AUSFÜHRLICH: [20210402-17:59:29] Finished Invoke-Command output.
AUSFÜHRLICH: [20210402-17:59:29] Configuring certificates.
AUSFÜHRLICH: [20210402-17:59:30] Begin Invoke-Command output:
AUSFÜHRLICH: [20210402-17:59:31] Finished Invoke-Command output.
AUSFÜHRLICH: [20210402-17:59:31] Adding Network Interfaces to network controller.
C:\Users\Walter\Desktop\SDN\SDNExpress.ps1 : Beim Ausführen des 'C:\Windows\system32\WindowsPowerShell\v1.0\Modules\networkcontroller\Microsoft.NetworkController.Powershell.dll'-Vorgangs für den serverseitigen Endpunkt
'https://sdn.home.arpa/networking/V1/operations/5a0f0ffc-b099-4ed7-b55f-93cbe776d1c0' wurde ein abbrechender Fehler ausgelöst.
In Zeile:1 Zeichen:1
+ .\SDNExpress.ps1 -ConfigurationDataFile .\LAN.psd1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (https://sdn.hom...5f-93cbe776d1c0:Uri) [SDNExpress.ps1], InvalidOperationException
+ FullyQualifiedErrorId : ODataEndpointProxyInvokeFailure,SDNExpress.ps1
This is in German, though it fails with an ODataEndpointProxyInvokeFailure for Adding Network Interfaces to network controller somewhere in https://github.com/microsoft/SDN/blob/master/SDNExpress/scripts/SDNExpressModule.psm1#L2151-L2173
As there is no other logging output generated here I can only speculate where it fails exactly.
This is the stack trace referenced in the operations-log:
System.ArgumentOutOfRangeException: Der Index lag außerhalb des Bereichs. Er darf nicht negativ und kleiner als die Sammlung sein.
Parametername: startIndex
bei System.ParseNumbers.StringToInt(String s, Int32 radix, Int32 flags, Int32* currPos)
bei System.Convert.ToUInt32(String value, Int32 fromBase)
bei Microsoft.Windows.Networking.NetworkController.RestApi.Common.NetworkInterfaceConverter.ConvertToWcf(NetworkInterfaceIpConfiguration ipConfig, List`1 vnetList)
bei System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
bei System.Linq.Buffer`1..ctor(IEnumerable`1 source)
bei System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
bei Microsoft.Windows.Networking.NetworkController.RestApi.Common.NetworkInterfaceConverter.ConvertToWcf(NetworkInterface networkInterface, List`1 vnetList)
bei Microsoft.Windows.Networking.NetworkController.RestApi.Tenant.Operations.PutNetworkInterfaceOperation.UpdateVsm(NetworkInterface networkInterface)
bei Microsoft.Windows.Networking.NetworkController.RestApi.Tenant.Operations.PutNetworkInterfaceOperation.RunBackgroundTaskInternal()
bei Microsoft.WindowsAzure.Networking.Nrp.Common.Utilities.Retry.<>c__DisplayClass4_0.<ExecuteWithRetriesTimeoutAsync>b__0()
bei Microsoft.WindowsAzure.Networking.Nrp.Common.Utilities.Retry.<ExecuteWithRetriesTimeoutAsync>d__5`1.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei Microsoft.WindowsAzure.Networking.Nrp.Common.Utilities.Retry.<ExecuteWithRetriesTimeoutAsync>d__4.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei Microsoft.Windows.Networking.NetworkController.RestApi.Common.Operations.PutResourceOperation`1.<<RunBackgroundTaskAsync>b__6_0>d.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei Microsoft.Windows.Networking.NetworkController.RestApi.Common.Operations.PutResourceOperation`1.<RunBackgroundTaskAsync>d__6.MoveNext()"
Its in German, translated would be Source index must not be negative or smaller than collection length.
I gathered the diagnostics information and the only thing failing here is
{
"resourceRef": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd",
"resourceId": "sdnGW01.ad.home.arpa_FrontEnd",
"name": "sdnGW01.ad.home.arpa_FrontEnd",
"id": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd",
"resourceGuid": "69ffb463-235e-41b7-a68b-89613fe6e905",
"etag": "W/\"b760f0f1-4856-4d53-85a9-40234c93fc20\"",
"groupName": "",
"subscriptionId": "defaultsubscription",
"fullName": [
"sdnGW01.ad.home.arpa_FrontEnd"
],
"lastOperationId": "f7120e0c-cbfb-4fa0-aa99-cc017a2ed1c0",
"instanceId": "44368d8a-b2dd-4ed7-b307-9c1bb12bc246",
"uniqueId": "44368d8a-b2dd-4ed7-b307-9c1bb12bc246",
"version": {
"major": 1,
"minor": 0
},
"lastOperationType": "Microsoft.Windows.Networking.NetworkController.RestApi.Tenant.Operations.PutNetworkInterfaceOperation",
"lastModifiedTime": "2021-04-02T15:40:26.2127973Z",
"createdTime": "2021-04-02T15:36:37.4004841Z",
"properties": {
"provisioningState": "Failed",
"previousPrivateMacAddress": "001DD8220008",
"ipConfigurations": [
{
"resourceRef": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd/ipConfigurations/FrontEnd",
"resourceId": "FrontEnd",
"name": "FrontEnd",
"id": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd/ipConfigurations/FrontEnd",
"resourceGuid": "30c571e8-2382-44dd-9151-c3f0eaf20167",
"etag": "W/\"b760f0f1-4856-4d53-85a9-40234c93fc20\"",
"groupName": "",
"subscriptionId": "defaultsubscription",
"fullName": [
"sdnGW01.ad.home.arpa_FrontEnd",
"FrontEnd"
],
"lastOperationId": "f7120e0c-cbfb-4fa0-aa99-cc017a2ed1c0",
"instanceId": "35f7f6b1-8a45-4112-9b07-353bbe7db706",
"uniqueId": "35f7f6b1-8a45-4112-9b07-353bbe7db706",
"version": {
"major": 1,
"minor": 0
},
"lastModifiedTime": "2021-04-02T15:40:26.2127973Z",
"createdTime": "2021-04-02T15:36:37.4004841Z",
"properties": {
"provisioningState": "Failed",
"privateIPAddress": "10.90.90.6",
"privateIPAllocationMethod": "Static",
"subnet": {
"resourceRef": "/logicalnetworks/HNVPA/subnets/10.90.90.0_24"
},
"previousSubnets": [],
"previousPublicIPAddresses": [],
"previousAllocationGoalsPrivateIPs": [],
"loadBalancerBackendAddressPools": [],
"loadBalancerInboundNatRules": [],
"isPrimary": false
}
}
],
"dnsSettings": {},
"privateMacAddress": "001DD8220008",
"privateMacAllocationMethod": "Static",
"serviceInsertionElements": [],
"portSettings": {
"provisioningState": "Succeeded",
"macSpoofingEnabled": "Disabled",
"arpGuardEnabled": "Disabled",
"dhcpGuardEnabled": "Disabled",
"stormLimit": 0,
"portFlowLimit": 0,
"iovWeight": 0,
"iovInterruptModeration": "Off",
"iovQueuePairsRequested": 0,
"vmqWeight": 100,
"qosSettings": {
"provisioningState": "Succeeded",
"outboundReservedValue": 0,
"outboundMaximumMbps": 0,
"inboundMaximumMbps": 0,
"enableHardwareLimits": false
}
},
"isHostVirtualNetworkInterface": false,
"isPrimary": false,
"isMultitenantStack": false,
"counters": []
}
So something with the ipConfigurations for sdnGW01_FrontEnd is wrong, but I can't for whatever reason determine what's wrong here. I deployed using the default MultiNodeSampleConfig with 3 NCs, 3 Mux and 3 GWs - won't work.
I tried 1 NC, 0 Mux, 2 Gw - won't work. 1 NC, 1 Mux, 2 GW (as minimally configured with SDNExpressGUI).
Checking on the Hyper-V Host I see PAhostVNic1 and PAhostVNic2 being created, but they fail to get an IP. Following configuration has been used to deploy SDN:
@{
ScriptVersion = "2.0"
VHDPath = "\\vault\VHDX"
VHDFile = "20303.1.amd64fre.fe_release.210226-1630_Server_ServerDatacenterCore_de-DE.vhdx"
JoinDomain = "ad.home.arpa"
SDNMacPoolStart = "00-11-22-00-01-00"
SDNMacPoolEnd = "00-11-22-00-01-FF"
ManagementSubnet = "10.159.226.0/24"
ManagementGateway = "10.159.226.1"
ManagementDNS = @("10.159.226.128")
DomainJoinUsername = "AD\AdminUser"
LocalAdminDomainUser = "AD\AdminUser"
RestName = "sdn.home.arpa"
NCs = @(
@{ComputerName='sdnNC01'; HostName='belsa'; ManagementIP='10.159.226.223'; MACAddress='001DD8220000'}
)
Muxes = @(
@{ComputerName='sdnMux01'; HostName='belsa'; ManagementIP='10.159.226.50'; MACAddress='001DD8220003'; PAIPAddress='10.90.90.4'; PAMACAddress='001DD8220004'}
)
Gateways = @(
@{ComputerName='sdnGW01'; HostName='belsa'; ManagementIP='10.159.226.21'; MACAddress='001DD8220007'; FrontEndIp='10.90.90.6'; FrontEndMac="001DD8220008"; BackEndMac="001DD8220009"},
@{ComputerName='sdnGW02'; HostName='eurodona'; ManagementIP='10.159.226.68'; MACAddress='001DD822000A'; FrontEndIp='10.90.90.7'; FrontEndMac="001DD822000B"; BackEndMac="001DD822000C"}
)
HyperVHosts = @(
"hv1.ad.home.arpa"
)
NCUsername = "AD\AdminUser"
PASubnet = "10.90.90.0/24"
PAGateway = '10.90.90.1'
PAPoolStart = '10.90.90.8'
PAPoolEnd = '10.90.90.13'
SDNASN = "65534"
Routers = @(
@{ RouterASN='64512'; RouterIPAddress='10.90.90.1'}
)
PrivateVIPSubnet = "10.0.0.0/24"
PublicVIPSubnet = "192.168.64.0/24"
PoolName = "DefaultAll"
GRESubnet = "192.168.0.0/24"
Capacity = 10000
VMMemory = 2GB
VMProcessorCount = 2
}
Yes, I'm using Server 2022, though it really doesn't make anything different if I'm using 2019 or even 2016 - it just doesn't get past Stage 5. Everything is to be configured without VLAN-ids. For testing purposes the Hyper-V-machine is virtualized, and the SDN infrastructure is virtualized in-parallel to the Hyper-V machine (Hosts belsa and eurodona host all SDN infrastructure and my testing Hyper-V machine). With the MultiNodeSampleConfig changed with VHD path, domain and user and only the management IPs changed it won't deploy either. So … I'm really at a loss here.
Not deploying the SDN Gateway is unfortunately no solution as then there is no way of adding the SDN endpoint to Windows Admin Center as it complains about invalid URIs.
To add to this:
- Windows Server 2022 (20303) and Server 2022 (20324)
- Windows Server 2019 (with latest cumulative patch)
Do not get past Stage 5 Adding Interfaces to network controller. Windows Server locale (de_DE, en_US) doesn't make any difference here.
This is the stack trace (this time in english):
curl.exe https://sdn.home.arpa/networking/V1/operations/b5a85276-16ff-4ed5-a84e-af664a907c22
{
"status": "Failed",
"error": {
"code": "InternalServerError",
"message": "An error occured.",
"innerError": "System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.\\r\\nParameter name: startIndex\\r\\n at System.ParseNumbers.StringToInt(String s, Int32 radix, Int32 flags, Int32* currPos)\\r\\n at System.Convert.ToUInt32(String value, Int32 fromBase)\\r\\n at Microsoft.Windows.Networking.NetworkController.RestApi.Common.NetworkInterfaceConverter.ConvertToWcf(NetworkInterfaceIpConfiguration ipConfig, List`1 vnetList)\\r\\n at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()\\r\\n at System.Linq.Buffer`1..ctor(IEnumerable`1 source)\\r\\n at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)\\r\\n at Microsoft.Windows.Networking.NetworkController.RestApi.Common.NetworkInterfaceConverter.ConvertToWcf(NetworkInterface networkInterface, List`1 vnetList)\\r\\n at Microsoft.Windows.Networking.NetworkController.RestApi.Tenant.Operations.PutNetworkInterfaceOperation.UpdateVsm(NetworkInterface networkInterface)\\r\\n at Microsoft.Windows.Networking.NetworkController.RestApi.Tenant.Operations.PutNetworkInterfaceOperation.RunBackgroundTaskInternal()\\r\\n at Microsoft.WindowsAzure.Networking.Nrp.Common.Utilities.Retry.<>c__DisplayClass4_0.<ExecuteWithRetriesTimeoutAsync>b__0()\\r\\n at Microsoft.WindowsAzure.Networking.Nrp.Common.Utilities.Retry.<ExecuteWithRetriesTimeoutAsync>d__5`1.MoveNext()\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Microsoft.WindowsAzure.Networking.Nrp.Common.Utilities.Retry.<ExecuteWithRetriesTimeoutAsync>d__4.MoveNext()\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Microsoft.Windows.Networking.NetworkController.RestApi.Common.Operations.PutResourceOperation`1.<<RunBackgroundTaskAsync>b__6_0>d.MoveNext()\\r\\n--- End of stack trace from previous location where exception was thrown ---\\r\\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\\r\\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\\r\\n at Microsoft.Windows.Networking.NetworkController.RestApi.Common.Operations.PutResourceOperation`1.<RunBackgroundTaskAsync>d__6.MoveNext()"
}
}
As well as the event viewer entry for the request:
Request: PUT https://sdn.home.arpa/networking/V4/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd
Host: sdn.home.arpa
User-Agent: Mozilla/5.0,(Windows NT; Windows NT 10.0; en-US),WindowsPowerShell/5.1.20324.3
Content-Length: 593
Content-Type: application/json; charset=utf-8
{"ResourceId":"sdnGW01.ad.home.arpa_FrontEnd","ResourceId:Key":"sdnGW01.ad.home.arpa_FrontEnd","Properties":{"PrivateMacAddress":"001DD8220008","PrivateMacAllocationMethod":"Static","IsHostVirtualNetworkInterface":false,"IsPrimary":false,"IsMultitenantStack":false,"IpConfigurations":[{"InstanceId":"00000000-0000-0000-0000-000000000000","ResourceId":"FrontEnd","Properties":{"IsPrimary":false,"PrivateIPAddress":"10.17.237.6","PrivateIPAllocationMethod":"Static","Subnet":{"ResourceRef":"/logicalnetworks/HNVPA/subnets/10.17.237.0_24","InstanceId":"00000000-0000-0000-0000-000000000000"}}}]}}
Response: x-ms-request-id: b5a85276-16ff-4ed5-a84e-af664a907c22
Azure-AsyncOperation: https://sdn.home.arpa/networking/V1/operations/b5a85276-16ff-4ed5-a84e-af664a907c22
Retry-After: 10
Content-Type: application/json; charset=utf-8
{
"resourceRef": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd",
"resourceId": "sdnGW01.ad.home.arpa_FrontEnd",
"etag": "W/\"d7676b38-b147-4947-ac99-beafde82b5cc\"",
"instanceId": "1c4792c4-1c05-4d81-a16d-6918226ff5e6",
"properties": {
"provisioningState": "Updating",
"ipConfigurations": [
{
"resourceRef": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd/ipConfigurations/FrontEnd",
"resourceId": "FrontEnd",
"etag": "W/\"d7676b38-b147-4947-ac99-beafde82b5cc\"",
"instanceId": "6ff8c8b0-e442-40a4-9d10-abf2339cea98",
"properties": {
"provisioningState": "Updating",
"privateIPAddress": "10.17.237.6",
"privateIPAllocationMethod": "Static",
"subnet": {
"resourceRef": "/logicalnetworks/HNVPA/subnets/10.17.237.0_24"
},
"loadBalancerBackendAddressPools": [],
"loadBalancerInboundNatRules": [],
"isPrimary": false
}
}
],
"dnsSettings": {},
"privateMacAddress": "001DD8220008",
"privateMacAllocationMethod": "Static",
"serviceInsertionElements": [],
"portSettings": {
"provisioningState": "Succeeded",
"macSpoofingEnabled": "Disabled",
"arpGuardEnabled": "Disabled",
"dhcpGuardEnabled": "Disabled",
"stormLimit": 0,
"portFlowLimit": 0,
"iovWeight": 0,
"iovInterruptModeration": "Off",
"iovQueuePairsRequested": 0,
"vmqWeight": 100,
"qosSettings": {
"provisioningState": "Succeeded",
"outboundReservedValue": 0,
"outboundMaximumMbps": 0,
"inboundMaximumMbps": 0,
"enableHardwareLimits": false
}
},
"isHostVirtualNetworkInterface": false,
"isPrimary": false,
"isMultitenantStack": false,
"counters": []
}
}
HttpStatusCode: Created
A CURL returns:
curl.exe https://sdn.home.arpa/networking/V4/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd/ipConfigurations/FrontEnd
{
"resourceRef": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd/ipConfigurations/FrontEnd",
"resourceId": "FrontEnd",
"etag": "W/\"c5aba806-97fb-479f-b3c4-375058b0b8a8\"",
"instanceId": "6ff8c8b0-e442-40a4-9d10-abf2339cea98",
"properties": {
"provisioningState": "Failed",
"privateIPAddress": "10.17.237.6",
"privateIPAllocationMethod": "Static",
"subnet": {
"resourceRef": "/logicalnetworks/HNVPA/subnets/10.17.237.0_24"
},
"loadBalancerBackendAddressPools": [],
"loadBalancerInboundNatRules": [],
"isPrimary": false
}
}
The HNVPA subnet 10.17.237.0/24 is successfully created:
curl.exe https://sdn.home.arpa/networking/V4/logicalnetworks/HNVPA/subnets/10.17.237.0_24
{
"resourceRef": "/logicalnetworks/HNVPA/subnets/10.17.237.0_24",
"resourceId": "10.17.237.0_24",
"etag": "W/\"d90152e6-0309-47a8-af74-2bd01d4dabd1\"",
"instanceId": "8789cf67-7000-4eb5-8edf-040a6d0ada24",
"properties": {
"provisioningState": "Succeeded",
"addressPrefix": "10.17.237.0/24",
"ipConfigurations": [
{
"resourceRef": "/networkInterfaces/sdnGW01.ad.home.arpa_FrontEnd/ipConfigurations/FrontEnd"
}
],
"networkInterfaces": [
{
"resourceRef": "/servers/bcc4364a-bacb-4671-97c0-55390695fae3/networkInterfaces/Default Switch"
},
{
"resourceRef": "/servers/1735ed87-4820-4915-af4f-dde857ff0773/networkInterfaces/Default Switch"
}
],
"gatewayPools": [],
"networkConnections": [],
"vlanID": "",
"ipPools": [
{
"resourceRef": "/logicalnetworks/HNVPA/subnets/10.17.237.0_24/ipPools/10.17.237.0_24",
"resourceId": "10.17.237.0_24",
"etag": "W/\"d90152e6-0309-47a8-af74-2bd01d4dabd1\"",
"instanceId": "c23dfe31-0834-4fab-89be-f377ba8e7e9d",
"properties": {
"provisioningState": "Succeeded",
"startIpAddress": "10.17.237.6",
"endIpAddress": "10.17.237.199"
}
}
],
"dnsServers": [],
"defaultGateways": [
"10.17.237.1"
],
"isPublic": false,
"usage": {
"numberOfIPAddresses": 194,
"numberofIPAddressesAllocated": 0,
"numberOfIPAddressesInTransition": 0
}
}
}
For testing purposes I just got a Windows Server 2016 image with latest updates applied, created two Hyper-V VMs (with nested virtualization) on two different Hyper-V hosts and Server 2016 deployment host.
Turns out: It doesn't work at all. Same issue again.
So:
Host 1 (Server 2022) -> Nested Hyper-V (HV1, 2016) -> Network Controller + Gateway 1
Host 2 (Server 2022) -> Nested Hyper-V (HV2, 2016) -> Mux + Gateway 2
Host 3 (Server 2022) -> SDN Deployment Host (Server 2016)
I even checked whether it has any effect on deployment if the nested hyper-v hosts are part of a Failover Cluster - no difference.
So conclusion:
It doesn't work, at all.
Neither Server 2022, 2019 nor 2016. Neither infrastructure VMs (SDN) on nested virtualization, nor siblings next to the Hyper-V VMs. It doesn't matter if they are part of a failover cluster. There is no difference on whether the Hyper-V Switch is added to the Hyper-V host with
New-VMSwitch -ManagementOS -SwitchName ""
orNew-VMSwitch -Management -SwitchName "" -Name "Management"
Not sure how was the configuration file generated, but the Gateway Frontend IPs (10.90.90.6, 10.90.90.7) are out of IP range of PA Pool (10.90.90.8 - 10.90.90.13). That should be the reason of error you get.
BTW. The PA Pool need to be larger as two PAs will be assigned for reach of the SDN Hosts after VM created and connected to Virtual Network
Gateways = @(
@{ComputerName='sdnGW01'; HostName='belsa'; ManagementIP='10.159.226.21'; MACAddress='001DD8220007'; FrontEndIp='10.90.90.6'; FrontEndMac="001DD8220008"; BackEndMac="001DD8220009"},
@{ComputerName='sdnGW02'; HostName='eurodona'; ManagementIP='10.159.226.68'; MACAddress='001DD822000A'; FrontEndIp='10.90.90.7'; FrontEndMac="001DD822000B"; BackEndMac="001DD822000C"}
)
HyperVHosts = @(
"hv1.ad.home.arpa"
)
NCUsername = "AD\AdminUser"
PASubnet = "10.90.90.0/24"
PAGateway = '10.90.90.1'
**PAPoolStart = '10.90.90.8'**
PAPoolEnd = '10.90.90.13'
Thanks for looking into this.
I originally created those via the SDNExpressUI, but then went back to just duplicating the MultiNode sample file and changing the configuration with PowerShell ISE. My last failed attempt was:
@{
ScriptVersion = "2.0"
VHDPath = "\\vault\VHDX"
VHDFile = "20303.1.amd64fre.fe_release.210226-1630_Server_ServerDatacenterCore_de-DE.vhdx"
JoinDomain = "ad.home.arpa"
SDNMacPoolStart = "00-11-22-00-01-00"
SDNMacPoolEnd = "00-11-22-00-01-FF"
ManagementSubnet = "10.159.226.0/24"
ManagementGateway = "10.159.226.1"
ManagementDNS = @("10.159.226.128")
DomainJoinUsername = "AD\Walter"
LocalAdminDomainUser = "AD\Walter"
RestName = "sdn.ad.home.arpa"
NCs = @(
@{
ComputerName = 'sdnNC01';
HostName = 'belsa';
ManagementIP = '10.159.226.223';
MACAddress = '00:1D:D8:22:00:00'
}
)
Muxes = @(
@{
ComputerName = 'sdnMux01';
HostName = 'olesia';
ManagementIP = '10.159.226.50';
MACAddress = '00:1D:D8:22:00:03';
PAIPAddress = '10.17.237.4';
PAMACAddress = '00:1D:D8:22:00:04'
}
)
Gateways = @(
@{
ComputerName = 'sdnGW01';
HostName = 'belsa';
ManagementIP = '10.159.226.21';
MACAddress = '00:1D:D8:22:00:07';
FrontEndIp = '10.17.237.6';
FrontEndMac = "00:1D:D8:22:00:08";
BackEndMac = "00:1D:D8:22:00:09"
},
@{
ComputerName = 'sdnGW02';
HostName = 'eurodona';
ManagementIP = '10.159.226.68';
MACAddress = '00:1D:D8:22:00:0A';
FrontEndIp = '10.17.237.7';
FrontEndMac = "00:1D:D8:22:00:0B";
BackEndMac = "00:1D:D8:22:00:0C"
}
)
HyperVHosts = @()
NCUsername = "AD\Walter"
PASubnet = "10.17.237.0/24"
PAGateway = '10.17.237.1'
PAPoolStart = '10.17.237.6'
PAPoolEnd = '10.17.237.199'
SDNASN = "65534"
Routers = @(
@{ RouterASN='64512'; RouterIPAddress='10.159.226.1'}
)
PrivateVIPSubnet = "10.0.0.0/24"
PublicVIPSubnet = "192.168.64.0/24"
PoolName = "DefaultAll"
GRESubnet = "192.168.0.0/24"
Capacity = 10000
VMMemory = 2GB
VMProcessorCount = 2
Fails with the same error in Adding network interface to network controller.
Is it showing same error code like before (index out of range)?. We need to confirm it from latest SDNExpress log. One more thing, I would assume you have wiped out any VMs (NC etc) before start next run of SDNExpress?
Yes, it is still Index out of range. I'm wiping out all VMs (NC01, Mux01, GW01, GW02) before starting again.
SDNExpress-20210409-074646.log (I did not capure the stack trace this time, though checked for the operations URL and it is the same stack trace as before).
Ok. There is one more error in the configuration. PA Subnet have no VLAN ID specified.
PASubnet = "10.17.237.0/24" PAVLANID = '11' # Missing VLAN ID here, 11 is an example, use yours. you can try 0 if there is no VLAN needed. PAGateway = '10.17.237.1' PAPoolStart = '10.17.237.6' PAPoolEnd = '10.17.237.199'
Okay, that did indeed the trick.
Though one thing thats missing for me: Windows Admin Center doesn't recognize the SDN setup on the host machines. It can validate the URL, and doesn't complain, but SDN is missing on the Cluster Manager.
Like: I have SDN Access control lists (1.21), SDN Gateway connections (1.22), SDN Load balancers (1.3), SDN logical networks (1.24), SDN monitoring (1.12), SDN Public IP Addresses (0.8), SDN Route tables (0.1) installed on WAC 2103, and it just doesn't want to show the SDN tab in the cluster connection window.
Do you think this is an issue with the Server 2022? I might give it a try with Server 2019 later …
Got around checking on all-Server 2019 infrastructure (latest patches, WAC, Hyper-V, Failover Cluster, and SDN Infrastructure) VMs are all Windows Server 2019 Datacenter Core, WAC has certificate of SDN REST Name in Local Machine Root store. And it won't allow me to manage the SDN infrastructure. Is there a hidden dependency on a specific deployment setup? (Does WAC require a specific number of NC, Mux and GW VMs?) (So I'd rule out Server 2022 compatibility). The only thing I'm seeing now is:
- Server version doesn't affect anything
- WAC 2103 is incapable of managing SDN (though does validate an SDN Rest Name sucessfully)
Okay … this might be a fluke, but based on the screenshot in this tech community-thread I figured I could give it a shot and add "SDN-Monitoring" to the URL and voilà it worked. Though I'm curious here: Why is the menu then left out of the sidebar for my HCI cluster?
For clarity: This is the state the WAC is at the moment:
SDN Monitoring works, but just isn't in the menu (WAC 2009 aswell as 2103).