TCP checksums not making it all the way through to the TCP stack?
erikarn opened this issue · 5 comments
Hi!
I have an odd problem in local testing - it seems hardware TCP checksums aren't making it up to the TCP stack in libuinet and it's doing software checksumming. Is this known?
[root@foo] ~# dtrace -n 'profile-1997 /arg1/ { @[execname, ustack()] = count(); } END { trunc(@, 3); }'
dtrace: description 'profile-1997 ' matched 2 probes
^C
CPU ID FUNCTION:NAME
15 2 :END
blockd
libc.so.7memcpy+0x20 blockd
netisr_dispatch_src+0x6c
blockdif_netmap_receive+0x332 blockd
pthread_start_routine+0xdb
libthr.so.30x8022954a5 166 blockd blockd
ether_nh_input+0x85
blockdnetisr_dispatch_src+0x6c blockd
if_netmap_receive+0x332
blockdpthread_start_routine+0xdb libthr.so.3
0x8022954a5
349
blockd
blockdin_cksumdata+0xdf blockd
in_cksum_skip+0x69
blockdtcp_input+0xc1 blockd
ip_input+0x8e
blockdnetisr_dispatch_src+0x6c blockd
ether_nh_input+0x61c
blockdnetisr_dispatch_src+0x6c blockd
if_netmap_receive+0x332
blockdpthread_start_routine+0xdb libthr.so.3
0x8022954a5
835
I don't think netmap
has any way to indicate that a packet has a hardware checksum yet; presumably the flags
field in struct netmap_slot
would need to track this, as well as possibly tracking it on the transmit side to indicate that the hardware needs to checksum on tx. In other similar systems, you get the actual bits from the hardware to report this, but netmap
's abstraction means that the relevant flags need to propagate by hand.
(And you'll definitely want to ensure they follow the packet from rx to tx in your application as well, especially if you're using VALE to connect multiple applications together.)
This is a netmap
issue and not a libuinet
one, unless you want a knob to tell libuinet
to assume all packets received from netmap
have had hardware checksum done. Which is not unreasonable for some hardware and configurations, but I'm glad it's not the default. I use netmap
with hardware that deliberately doesn't do any packet checking routinely.
Yeah, I dug into it a bit more and I agree.
I think it'd be useful to get TCP checksums propagated down/up the stack but yes, we'd have to start by extending netmap to include all that metadata.
Do you think you'd find value in a mode whereby libuinet
could be configured to flag all packets received from netmap
as having valid checksums? Or should this be closed until netmap
grows support?
I think we should keep this open for the time being. I would like to see
netmap grow hardware transmit/receive checksum support so libuinet doesn't
have to do it. Otherwise a whole lot of CPU time is spent calculating
checksums.
There is (unreleased) code in netmap-fwd to do this. I'll get a copy to Patrick.