How Serverless Containers Achieve Service Discovery Without kube-proxy Using eBPF

Serverless Container Service Discovery

In September 2020 UCloud launched the Serverless container product Cube, offering VM‑level security isolation, lightweight resource usage, second‑level startup, highly automated scaling, and simple usability.

By leveraging Virtual Kubelet technology, Cube seamlessly integrates with UCloud's managed Kubernetes service UK8S, greatly enhancing the elasticity of Kubernetes clusters. Each Cube instance appears as a pod on a virtual node within the cluster.

Why Not Use kube-proxy?

kube‑proxy implements service traffic load balancing in Kubernetes, maintaining Service‑to‑Endpoint mappings and generating forwarding rules via userspace, iptables, or ipvs (userspace is deprecated).

Deploying kube‑proxy inside Cube's lightweight VMs is problematic for several reasons:

The Go‑compiled binary is large (≈37 MB after stripping), exceeding the <10 MB size target for serverless VMs.

Performance overhead of Go compared to C/C++ or Rust is undesirable for the fine‑grained resource quotas of Cube (e.g., 0.5 C, 500 MiB).

ipvs, while stable for large service scales, has issues such as longer processing paths and imperfect handling of NAT and conn_reuse_mode.

iptables suffers from limited extensibility, slow updates, O(n) rule lookup, and lacks full_random port selection in older kernels, affecting SNAT performance under high concurrency.

What Can eBPF Bring to Container Networking?

eBPF is a revolutionary Linux technology that allows developers to load sandboxed programs into the kernel at runtime without modifying kernel source or loading modules. User‑space programs interact with eBPF via the bpf(2) syscall and BPF maps.

eBPF programs can attach to various hook points (xdp, tc, qdisc, socket, kprobes, uprobes, etc.) to control and modify network packets, enabling load balancing, security policies, and observability.

Cilium introduced eBPF to Kubernetes, eventually allowing it to replace kube‑proxy entirely. For ClusterIP, the client performs DNAT using an eBPF map that stores Frontend‑to‑Backend address mappings.

DNAT can be applied at the tc egress hook or, on cgroup2‑enabled kernels, via the BPF_PROG_TYPE_CGROUP_SOCK_ADDR hook, modifying destination addresses before packet generation.

Implementation of ClusterIP

For TCP and connected UDP flows, a single forward conversion changes the destination address to the target pod. For connectionless UDP, an additional reverse conversion (SNAT) restores the source address to the ClusterIP, ensuring correct reception of responses.

This approach provides user‑transparent address translation: applications think they connect to the Service IP, while the actual TCP connection targets the backend pod directly, eliminating the need for conntrack and reducing the forwarding path compared to ipvs or iptables.

Cube's Service Discovery

Cube launches a lightweight agent called cproxy for each serverless container group that requires ClusterIP access. Built in Rust, cproxy's binary is under 10 MiB and runs with lower overhead than kube‑proxy, using the cgroup2 socket hook for ClusterIP forwarding.

Performance tests using wrk with 2000 concurrent HTTP short connections show that cproxy consistently outperforms ipvs and iptables across both small (10 services) and large (5000 services) service counts.

UCloud Embraces eBPF

eBPF is reshaping the cloud‑native ecosystem. UCloud plans to tightly integrate its managed Kubernetes service UK8S with the Serverless Cube product, further exploring eBPF applications in networking, load balancing, and monitoring to provide better observability, troubleshooting, and tuning capabilities for users.