(Go language Gopher flying on a eBPF bee and carrying a switch, created by Microsoft Designer DALL-E 3)
pktstat-bpf is a simple replacement for ncurses/libpcap-based pktstat, using Linux eBPF (extended Berkeley Packet Filter) program, allowing packet statistics gathering even under very high traffic volume conditions, typically several million packets per second even on an average server. In this scenario (high volume, DoS attacks etc.) typically regular packet capture solutions start being unreliable due to increasing packet loss.
By default it uses TC (Traffic Control) eBPF hooks with TCX attaching requiring at minimum Linux kernel v6.6 for both ingress and egress traffic statistics. Alternatively it can switch to XDP (eXpress Data Path) hook but with a consequence of losing egress statistics since XDP works only in ingress path. XDP mode due to XDP program to network interface attaching calls requires at minimum Linux kernel v5.9. Some distributions might have backported XDP/TC patches (notable example is Red Hat Enterprise Linux kernel) and eBPF program might work on older kernels too.
At the end of the execution program will display per-IP and per-protocol statistics sorted by per-connection bps, packets and (source-IP:port, destination-IP:port) tuples.
Program consists of the eBPF code in C and the pure-Go userland Golang program that parses and outputs final IP/port/protocol/bitrate statistics. Go part of the program uses wonderful cillium/ebpf library to load and run eBPF program, interfacing with eBPF map.
Loading eBPF program typically requires root privileges and in our specific case pointer arithmetics in eBPF code causes eBPF verifier to explicitly deny non-root use.
Typically BPF JIT (Just in Time compiler) should be enabled for best performance:
echo 1 > /proc/sys/net/core/bpf_jit_enableIn case of XDP, not all NIC drivers support Native XDP (XDP program is loaded by NIC driver with XDP support as part of initial receive path and most common 10G drivers already support this) or even Offloaded XDP (XDP program loads directly on NIC with hardware XDP support and executes without using CPU), causing kernel to fallback on Generic XDP with reduced performance. Generic XDP does not require any special support from NIC drivers, but such XDP happens much later in the networking stack and in such case performance is more or less equivalent to TC hooks.
The following table maps features, requirements and expected performance for described modes:
| Capture type | Ingress | Egress | Performance | Kernel required | SmartNIC required |
|---|---|---|---|---|---|
| Generic PCAP | Yes | Yes | Low | Any | No |
| AF_PACKET | Yes | Yes | Medium | v2.2 | No |
| TC | Yes | Yes | High | v6.6 | No |
| XDP Generic | Yes | No | High | v5.9 | No |
| XDP Native | Yes | No | Very high | v5.9 | No |
| XDP Offloaded | Yes | No | Wire speed | v5.9 | Yes |
A list of XDP compatible drivers follows (and it is not necessarily up-to-date):
NAME
pktstat-bpf
FLAGS
-?, --help display help
-j, --json if true, output in JSON format
-x, --xdp if true, use XDP instead of TC (this disables egress statistics)
--version display program version
-i, --iface STRING interface to read from (default: anpi5)
--xdp_mode STRING XDP attach mode (auto, generic, native or offload; native and offload require NIC driver support) (default: auto)
-t, --timeout DURATION timeout for packet capture (default: 1h0m0s)It is possible to specify interface with --iface.
Timeout --timeout will stop execution after a specified time, but it is also possible to interrupt program with Ctrl C, SIGTERM or SIGINT.
With --json it is possible to get traffic statistics in JSON format.
With --xdp program will switch from TC eBPF mode to XDP eBPF mode, working in even more high-performance mode however this will disable all egress statistics. On program exit it is also possible to get an interface reset, so it is best to use this program inside of screen or tmux.
Additionally it is possible to change XDP attach mode with --xdp_mode from auto (best-effort between native and generic) to native or offload, for NIC drivers that support XDP or even NICs that have hardware XDP support.