Unexpected route decision after accessing nodeport across nodes

[mars1024]

Bug Report

Type: bug report

What happened

node1: 11.166.83.9
node2: 11.166.83.20
svc: nodePort 30991, only have one pod backend, pod ip: 100.88.253.95 on node2
network: vlanid 701
master interface: bond0

test network access: from node1 to node2:30991

#curl 11.166.83.20:30991
404 page not found

tcpdump on interface bond0.701 and got unexpected traffic

#tcpdump -nv -i bond0.701 host 11.166.83.9 and host 11.166.83.20 and port 30991
tcpdump: listening on bond0.701, link-type EN10MB (Ethernet), capture size 262144 bytes
16:09:03.511036 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    11.166.83.20.30991 > 11.166.83.9.37957: Flags [S.], cksum 0xbd97 (incorrect -> 0xcde4), seq 2388492280, ack 4182406673, win 28960, options [mss 1460,sackOK,TS val 522611790 ecr 2192028254,nop,wscale 7], length 0
16:09:03.511207 IP (tos 0x0, ttl 63, id 5047, offset 0, flags [DF], proto TCP (6), length 52)
    11.166.83.20.30991 > 11.166.83.9.37957: Flags [.], cksum 0xbd8f (incorrect -> 0x6c9c), ack 83, win 227, options [nop,nop,TS val 522611790 ecr 2192028254], length 0
16:09:03.511374 IP (tos 0x0, ttl 63, id 5048, offset 0, flags [DF], proto TCP (6), length 228)
    11.166.83.20.30991 > 11.166.83.9.37957: Flags [P.], cksum 0xbe3f (incorrect -> 0x4865), seq 1:177, ack 83, win 227, options [nop,nop,TS val 522611790 ecr 2192028254], length 176
16:09:03.511649 IP (tos 0x0, ttl 63, id 5049, offset 0, flags [DF], proto TCP (6), length 52)
    11.166.83.20.30991 > 11.166.83.9.37957: Flags [F.], cksum 0xbd8f (incorrect -> 0x6be9), seq 177, ack 84, win 227, options [nop,nop,TS val 522611790 ecr 2192028255], length 0

We can see the reply network packets above, all went into bond0.701 after routing.
But, the packets between nodes are expected to go through bond0.

What you expected to happen

All reply network packets go through bond0, but not bond0.701.

How to reproduce it (as minimally and precisely as possible)

Access nodeport across nodes on underlay network mode.

Anything else we need to know?

Sometimes, although the network traffic is unexpected but it works, because container network router received these network packets and forward to nodes, if rp_filter and source address detection are both disabled.

Environment

• hybridnet version: v0.3.2
• OS (e.g. cat /etc/os-release):
• Kernel (e.g. uname -a):
• Kubernetes version:
• Install tools:
• Others: The Kubernetes svc implementation is with kube-proxy.

[hhyasdf]

This is because kernel will not trigger the rerouting process after traffic is SNATed in netfilter, which is unexpected :)

I'm working on it to adapt to the nodeport svc of kube-proxy.