Network endpoints created by CNI NAT plugin is not attached to container
yldoge opened this issue · 0 comments
yldoge commented
The ipconfig inside the container returns nothing.
Beginning state of the network:
After code run:
Should the 2 extensions be enabled?
PS C:\Windows\system32> Get-HNSNetwork
ActivityId : 2D5B4A5C-40C8-4707-9DD9-62AA866A3B7C
AdditionalParams :
CurrentEndpointCount : 1
Extensions : {@{Id=E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A; IsEnabled=False; Name=Microsoft Windows Filtering Platform},
@{Id=F74F241B-440F-4433-BB28-00F89EAD20D8; IsEnabled=False; Name=Microsoft Azure VFP Switch Extension},
@{Id=430BDADD-BAB0-41AB-A369-94B67FA5BE0A; IsEnabled=True; Name=Microsoft NDIS Capture}}
Flags : 8
Health : @{LastErrorCode=0; LastUpdateTime=133318808183134023}
ID : 85D61148-49EF-40BB-A450-3BEBE951D838
IPv6 : False
LayeredOn : 24DDF755-8077-4CAD-BEE4-6D6B6B4E015B
MacPools : {@{EndMacAddress=00-15-5D-2B-FF-FF; StartMacAddress=00-15-5D-2B-F0-00}}
MaxConcurrentEndpoints : 1
Name : nat
NatName : NAT48C0DEC2-ED26-4DC5-A408-9B510A985ABC
Policies : {@{Type=VLAN; VLAN=1}}
State : 1
Subnets : {@{AdditionalParams=; AddressPrefix=192.168.100.0/24; Flags=0; GatewayAddress=192.168.100.1; Health=;
ID=8350EC42-2087-4941-A535-BB4E118B8797; IpSubnets=System.Object[]; ObjectType=5; Policies=System.Object[]; State=0}}
TotalEndpoints : 1
Type : NAT
Version : 55834574851
Resources : @{AdditionalParams=; AllocationOrder=2; Allocators=System.Object[]; CompartmentOperationTime=0; Flags=0; Health=;
ID=2D5B4A5C-40C8-4707-9DD9-62AA866A3B7C; PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
parentId=1826BFFB-7D84-461D-B81E-67D7C1BD94B7}
PS C:\Windows\system32> Get-NetIPAddress | Format-Table
ifIndex IPAddress PrefixLength PrefixOrigin SuffixOrigin AddressState PolicyStore
------- --------- ------------ ------------ ------------ ------------ -----------
18 fe80::2026:12f0:fe15:51c8%18 64 WellKnown Link Preferred ActiveStore
11 fe80::3e4b:b425:85f5:4a92%11 64 WellKnown Link Preferred ActiveStore
1 ::1 128 WellKnown WellKnown Preferred ActiveStore
18 192.168.100.1 24 Manual Manual Preferred ActiveStore
11 10.44.28.245 23 Dhcp Dhcp Preferred ActiveStore
1 127.0.0.1 8 WellKnown WellKnown Preferred ActiveStore
Should the generated namespace policy be my portMapping rules, instead of empty?
PS C:\Windows\system32> Get-HnsNamespace
ActivityId : F1173E93-F496-4595-B99C-E42ED06A2D82
AdditionalParams :
CompartmentGuid : 79D9FA50-97D7-420B-AF27-864C96489ECC
CompartmentId : 2
Containers : {test-container}
Flags : 0
Health : @{LastErrorCode=0; LastUpdateTime=133318808092551847}
ID : B857CF74-2286-416E-A4FE-9B174FB4FDA9
IsDefault : False
Policies : {}
ResourceList : {@{Data=; Type=Endpoint}}
SchemaVersion : @{Major=0; Minor=0}
State : 3
Version : 55834574851
Resources : @{AdditionalParams=; AllocationOrder=1; Allocators=System.Object[]; CompartmentOperationTime=0; Flags=0; Health=;
ID=F1173E93-F496-4595-B99C-E42ED06A2D82; PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0}
PS C:\Windows\system32> Get-VMSwitch
Name SwitchType NetAdapterInterfaceDescription
---- ---------- ------------------------------
nat Internal
PS C:\Windows\system32> Get-HnsEndPoint
ID : 64dd3052-e077-4d11-951c-8fdb73f2ebd5
Name : test-container_nat
Version : 55834574851
AdditionalParams :
Resources : @{AdditionalParams=; AllocationOrder=4; Allocators=System.Object[]; CompartmentOperationTime=0; Flags=0; Health=;
ID=6FB7AB65-8CB2-4623-B444-1ED5425A2CFC; PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
parentId=2D5B4A5C-40C8-4707-9DD9-62AA866A3B7C}
State : 2
VirtualNetwork : 85d61148-49ef-40bb-a450-3bebe951d838
VirtualNetworkName : nat
Policies : {@{ExternalPort=8886; InternalPort=8888; Protocol=TCP; Type=NAT}}
MacAddress : 00-15-5D-2B-F8-1C
IPAddress : 192.168.100.170
PrefixLength : 24
GatewayAddress : 192.168.100.1
IPSubnetId : 2b2fda3e-a14e-439d-8de8-92ff2395ef31
DNSServerList : 10.50.4.32,10.50.4.33,10.48.4.1
DNSSuffix :
Namespace : @{ID=b857cf74-2286-416e-a4fe-9b174fb4fda9}
SharedContainers : {test-container}
PS C:\Windows\system32> ping 192.168.100.170
Pinging 192.168.100.170 with 32 bytes of data:
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Ping statistics for 192.168.100.170:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Is able to ping the endpoint from host namespace
TCP 127.0.0.1:53562 Dev0027986:53563 ESTABLISHED
TCP 127.0.0.1:53563 Dev0027986:53562 ESTABLISHED
TCP 127.0.0.1:53616 Dev0027986:53619 ESTABLISHED
TCP 127.0.0.1:53619 Dev0027986:53616 ESTABLISHED
TCP [::1]:53918 Dev0027986:5985 TIME_WAIT
TCP [::1]:53921 Dev0027986:5985 TIME_WAIT
PS C:\Windows\system32> curl http://localhost:8886
curl : Unable to connect to the remote server
At line:1 char:1
+ curl http://localhost:8886
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
My external port is 8886, no port 8886 is established.
My CNI config:
{
"cniVersion": "0.2.0",
"name": "nat",
"type": "nat",
"master": "Ethernet",
"ipam": {
"subnet": "192.168.100.0/24",
"routes": [
{
"gateway": "192.168.100.1"
}
]
},
"capabilities": {
"portMappings": true,
"dns": true
}
}
CNI plugin log:
My code logic:
containerId := "test-container"
l, err := gocni.New(
gocni.WithPluginConfDir(cniConfDir),
gocni.WithPluginDir(cniPluginDir),
)
l.Load(gocni.WithDefaultConf)
netNs, err := netns.NewNetNS("")
result, err := l.Setup(ctx, containerId, netNs.GetPath(), nsOpts...)
// ============================ create container ============================
img, err := h.conn.GetImage(ctx, image)
c, err := h.conn.NewContainer(ctx, containerId,
containerd.WithImage(img),
containerd.WithNewSpec(
oci.WithDefaultSpec(),
oci.WithImageConfig(img),
oci.WithHostname("test-container-hostname"),
oci.WithMounts(mounts),
oci.WithWindowNetworksAllowUnqualifiedDNSQuery(),
oci.WithWindowsIgnoreFlushesDuringBoot(),
oci.WithWindowsNetworkNamespace(netNs.GetPath()),
),
containerd.WithNewSnapshot(containerId, img),
)
task, err := c.NewTask(ctx, cio.NewCreator(cio.WithStdio))
task.Start(ctx)
Containerd version:
Client:
Version: v1.7.2
Revision: 0cae528dd6cb557f7201036e9f43420650207b58
Go version: go1.20.4
Server:
Version: v1.7.2
Revision: 0cae528dd6cb557f7201036e9f43420650207b58
UUID: c775f801-1980-4709-82da-fd2a591e7be3
OS: Windows Server 2022