At netdev layer of Linux networking stack, there are two tracepoints to measure packet's latency:
netif_receive_skb: This tracepoint is triggered when a packet is received by the network device driver.net_dev_xmit: This tracepoint is triggered when a packet is transmitted by the network device driver.
$ ./skbdist -h
Usage: ./skbdist [options] [pcap-filter]
Available pcap-filter: see "man 7 pcap-filter"
Available options:
--dist-cpu Measure distribution of CPU
--dist-latency Measure distribution of skb latency
--dist-queue Measure distribution of queue
--dist-skblen Measure distribution of skb length
-i, --interface string Interface to filter packets, all interfaces if not specified
-n, --non-core Run with non-core bpf [TODO]NOTE: If to use pcap-filter, src/dst should not be used in it.
To measure the response latency of a TCP port, you can use the following command:
$ sudo ./skbdist --dist-cpu --dist-queue --dist-skblen --dist-latency tcp port 8080
2024/07/15 13:41:27 Attached tracepoint/net/netif_receive_skb
2024/07/15 13:41:27 Attached tracepoint/net/net_dev_xmit
Ctrl+C to show results..
^C
Send SKB lengths (total 5 pkts) :
byte : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 3 |****************************************|
128 -> 255 : 1 |************* |
256 -> 511 : 0 | |
512 -> 1023 : 1 |************* |
Receive SKB lengths (total 6 pkts) :
byte : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 4 |****************************************|
64 -> 127 : 1 |********** |
128 -> 255 : 1 |********** |
Recv CPU's distribution (total 6 pkts) :
CPU 6: 6
Xmit CPU's distribution (total 5 pkts) :
CPU 3: 2
CPU 6: 3
Recv queue distribution (total 6 pkts) :
Queue 0: 6
192.168.241.133:8080 -> 192.168.241.1:61645 (TCP) (total 5 records) :
µs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 1 |****************************************|
4 -> 7 : 0 | |
8 -> 15 : 1 |****************************************|
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 1 |****************************************|
128 -> 255 : 1 |****************************************|
256 -> 511 : 0 | |
512 -> 1023 : 0 | |
1024 -> 2047 : 0 | |
2048 -> 4095 : 0 | |
4096 -> 8191 : 0 | |
8192 -> 16383 : 1 |****************************************|skbdist requires the following dependencies:
clangandllvm: To compile eBPF program.libpcap.a: To inject pcap-filter to eBPF program.
Build and run:
go generate
go build -v
./skbdist -h- Support non-CORE bpf on old kernels
- Support filtering inner-VxLAN packets
- elibpcap: A really cool library to inject pcap-filter into eBPF programs.
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.