eBPF XDP Tutorial: Packet Parsing, Rewriting & Redirection

packet01: Packet Parsing

Network packet parsing is tedious and cumbersome.

Directly using xdp-loader to load the example eBPF program shows:

Couldn't attach XDP program on iface 'veth-packet01': Permission denied(-13)

This error is puzzling; one might think it's a permission issue.

Adding the -vv parameter clarifies:

Validating xdp_parser_func() func#0...
0: R1=ctx() R10=fp0
; int xdp_parser_func(struct xdp_md *ctx) @ xdp_prog.bpf.c:56
0: (bf) r6 = r1 ; R1=ctx() R6_w=ctx()
1: (b7) r1 = 2 ; R1_w=2
; void *data_end = (void *)(long)ctx->data_end; @ xdp_prog.bpf.c:58
2: (61) r3 = *(u32 *)(r6 +4) ; R3_w=pkt_end() R6_w=ctx()
; void *data = (void *)(long)ctx->data; @ xdp_prog.bpf.c:59
3: (61) r2 = *(u32 *)(r6 +0) ; R2_w=pkt(r=0) R6_w=ctx()
; if (nh->pos + 1 > data_end) @ xdp_prog.bpf.c:32
4: (bf) r4 = r2 ; R2_w=pkt(r=0) R4_w=pkt(r=0)
5: (07) r4 += 1 ; R4_w=pkt(off=1,r=0)
6: (2d) if r4 > r3 goto pc+3 ; R3_w=pkt_end() R4_w=pkt(off=1,r=1)
; return eth->h_proto; /* network-byte-order */ @ xdp_prog.bpf.c:38
7: (69) r2 = *(u16 *)(r2 +12)
invalid access to packet, off=12 size=2, R2(id=0,off=12,r=1)
R2 offset is outside of the packet
processed 8 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
 libbpf: prog 'xdp_parser_func': failed to load: -13
 libbpf: failed to load object 'xdp_prog.bpf.o'
 libxdp: Failed to load program xdp_parser_func: Permission denied
Couldn't attach XDP program on iface 'veth-packet01': Permission denied(-13)

The eBPF program verifier can even check boundary logic. Amazing!

packet02: Packet Rewriting

Rewriting port numbers is relatively simple, but the author encountered a few issues during the experiment.

1. Observing with nc

For observing results, besides tcpdump, you can also use nc to receive UDP packets: nc -ul6 1999 This is relatively more intuitive.

2. Debugging with bpf_printk

While writing code, the author made a mistake by blindly adding bpf_htons, and eventually used bpf_printk to troubleshoot. Indeed, logging is the most "universal" debugging method.

By the way, tail the log like this:

cat /sys/kernel/debug/tracing/trace_pipe

3. --pin-path for Maps Path

The experiment guide uses xdp_loader from the basic course to load the XDP program because its pinned BPF maps path format matches that polled by xdp_stats.

Actually, xdp-loader from xdp-tools can also specify the path under /sys/fs/bpf/ via the --pin-path parameter.

However, there's a small issue: xdp-loader does not unpin custom paths on unload, requiring manual cleanup.

packet03: Packet Redirection

Packet redirection gets interesting! There's a small detail to explain.

Note that in order for the transmit and/or redirect functionality to work, all involved devices should have an attached XDP program, including both veth peers.

veth devices are paired virtual devices, like a pipe: one end sends, the other receives. When an XDP frame (containing the raw packet and XDP metadata) is sent to the receiving end of a veth device via XDP_REDIRECT or XDP_TX, the veth kernel driver by default does not inject it back into the network processing path.

The veth driver only changes its behavior when it detects an XDP program attached to the receiving veth device, to correctly handle these special XDP frames: that is, understand and process the accompanying XDP metadata and place the packet into the normal receive path.

There is an exception: xdpgeneric mode, where the XDP program runs higher in the network stack; the packet has already been preliminarily processed by the network stack and converted into an sk_buff structure. Redirection in this mode is typically achieved by modifying the sk_buff 's destination device field, and the subsequent kernel network stack naturally handles this modification without requiring the receiving driver to specially process XDP frames.

Reference:

https://github.com/xdp-project/xdp-tutorial