Can eBPF Replace Netfilter? Boosting Kubernetes Network Performance

Background

Kubernetes networking consists of three parts: pod network, service network, and network policy. The pod network forms a flat L3 network that underpins the other two. Service network adds load‑balancing on top of the pod network, while network policy provides stateful firewall‑style access control.

Performance Problems of Netfilter‑Based Service Network and Network Policy

Implemented on the kernel Netfilter framework (conntrack, iptables, NAT, IPVS).

Linear iptables rule structure leads to slow updates and matching.

Conntrack performance degrades quickly under high concurrency, and connection tables can become full.

These issues limit scalability and high‑performance scenarios for large Kubernetes clusters.

Pod Network Implementation Types

L3‑based solutions (routing, bridge) retain full L3 processing, compatible with existing service network and policy implementations, but suffer from higher latency.

Bypass‑L3 solutions (IPVlan, MacVlan, direct‑attach) have simple datapaths and better performance, but are harder to integrate with service network and policy features.

Need for a Lightweight Non‑Netfilter Solution

To support high‑performance pod networks such as IPVlan, MacVlan, and direct‑attach, a new implementation must provide service network and network policy capabilities without the performance and scalability drawbacks of Netfilter.

eBPF Overview

eBPF introduces a register‑based virtual machine inside the kernel with its own 64‑bit instruction set. Programs are loaded via system calls, verified for safety, and executed on relevant kernel events, allowing programmable data‑plane processing.

tc‑ebpf Design

Hooks at the TC layer, bypassing the kernel L3 stack, making it easy to support IPVlan, MacVlan, and direct‑attach networks.

Full read/write access to skb_buff with rich helper functions, simplifying complex datapath logic.

Rule matching and updates use hash tables, offering higher performance and better scalability than linear iptables rules.

Datapath is programmable; updates and optimizations can be applied without rebooting or upgrading the kernel.

Implementation Modules

Conntrack : Core module for both egress and ingress traffic, maintaining connection state.

LoadBalance + ReverseNat : Selects an endpoint for egress traffic, syncs load‑balancing info to conntrack, and performs reverse NAT for ingress traffic to rewrite the source IP to the cluster IP.

Policy : Applies user‑defined policy rules based on conntrack connection states.

Comparison with Existing Solutions

Cilium : Uses eBPF to implement pod network, service network, and network policy, but its current modes do not fit well with public‑cloud scenarios.

Cloud‑provider network products : Provide rich features and SLAs but focus on IaaS layer; creation speed and quota limits hinder elastic cloud‑native workloads.

Terway with eBPF Solution

Terway creates a pod network on Alibaba Cloud VPC using ENI multi‑IP and operates as the first CNI in a CNI‑chain. Cilium is added as the second CNI to provide high‑performance service network and network policy. The chain combines Terway’s L2 pod network with tc‑ebpf datapath for service networking and policy enforcement.

Performance Evaluation

Benchmarks compare the Terway‑with‑eBPF mode against three existing Terway modes (policy‑routing, IPVlan, ENI) and against kube‑proxy (iptables and IPVS).

Pod network : eBPF mode is within 2 % of IPVlan performance, 11 % faster than policy‑routing.

Scalability : When pod count grows from 3 to 156, IPVlan and eBPF drop ~13 % while policy‑routing drops ~23 %.

Service network : Compared to kube‑proxy iptables, short‑connection latency improves ~32 % and long‑connection ~19 %; compared to kube‑proxy IPVS, short‑connection improves ~60 % and long‑connection ~50 %.

Service scalability : With services increasing from 1 to 5000, eBPF and IPVS maintain similar performance (≈3 % drop at 5000 services), while iptables drops ~60 % for short connections.

Network policy : With 1 pod and 1 policy, eBPF outperforms policy‑routing by ~32 % (short) and ~16 % (long); with 120 pods/policies, short‑connection improves 19 % and long‑connection 24 %.

Conclusion and Future Work

The eBPF‑based L2 implementation delivers noticeable performance gains over Netfilter‑based solutions, yet the conntrack module still incurs non‑trivial overhead. Ongoing research explores eliminating conntrack by leveraging socket‑level state or implementing a lightweight eBPF‑based conntrack to further improve performance and scalability.