How Flannel’s VXLAN Overlay Enables Seamless Pod Communication in Kubernetes

Flannel is one of the CNI network plugins for Kubernetes, implementing a host overlay network. It supports multiple forwarding modes; this article focuses on the UDP VXLAN mode.

Flannel Features

Provides a unique virtual IP address to each Docker container across all nodes in the cluster.

Creates an overlay network that forwards packets unchanged to the destination container, encapsulating and decapsulating packets at the edges.

Creates a virtual interface flannel0 that receives data from the Docker bridge and forwards it using VXLAN routing.

Uses etcd to keep configuration consistent across nodes; each node watches etcd for changes.

Component Explanation

Cni0

: a bridge device; each pod creates a veth pair, one end in the pod (eth0) and the other attached to Cni0. Traffic from the pod’s eth0 is sent to Cni0. Flannel.1: the overlay device that handles VXLAN encapsulation and decapsulation, sending traffic between nodes as tunnels. Flanneld: the agent running on each host; it obtains a subnet from the cluster address space and assigns IPs to containers, while listening to the Kubernetes datastore to provide MAC and IP information for encapsulation.

Pod Communication Flow Across Nodes

Pod generates data and sends it to Cni0 based on its routing table.

Cni0 forwards the data to the tunnel device flannel.1.

Flannel.1 looks up the destination IP, obtains the remote tunnel endpoint information from flanneld, and encapsulates the packet.

Flannel.1 sends the encapsulated packet to the remote node, where the overlay packet is received, decapsulated, and passed to the remote flannel.1.

Flannel.1 matches the packet to the routing table and forwards it to Cni0.

Cni0 matches its routing table and forwards the packet to the appropriate bridge port.

The test cluster uses POD CIDR 172.20.0.0/16. The following examples illustrate routing tables and host routes for two pods (pod1 on node 10.19.114.100 and pod2 on node 10.19.114.101).

10.19.114.100 - pod1 route
# kubectl -n stack exec -it api-0 -- bash
ip route show
default via 172.20.0.1 dev eth0
172.20.0.0/24 dev eth0 proto kernel scope link src 172.20.0.73
172.20.0.0/16 via 172.20.0.1 dev eth0

10.19.114.101 - pod2 route
# kubectl -n stack exec -it redis-64c6c549ff-5plcq -- bash
ip route show
default via 172.20.1.1 dev eth0
172.20.0.0/16 via 172.20.1.1 dev eth0
172.20.1.0/24 dev eth0 proto kernel scope link src 172.20.1.11

Host routing tables show how traffic is forwarded to the flannel.1 device, which then encapsulates the packet and uses the FDB (Forwarding Database) to resolve the remote VTEP IP based on MAC addresses.

# ip neigh | grep 172
172.20.2.0 dev flannel.1 lladdr 82:c4:0e:f2:00:6f PERMANENT
172.20.1.0 dev flannel.1 lladdr 42:6e:8b:9b:e2:73 PERMANENT

When the kernel cannot find a matching entry in the FDB, it generates an “L2 MISS” event that flanneld handles by querying etcd for the public IP of the target node and updating the FDB.

VXLAN forwarding relies on the FDB to map destination MAC addresses to VTEP IPs, after which the outer encapsulation is performed.

# /sbin/bridge fdb show dev flannel.1
42:6e:8b:9b:e2:73 dst 10.19.114.101 self permanent
ba:8b:ce:f3:b8:51 dst 10.19.114.101 self permanent

The bridge cni0 connects pod veth pairs to the host, allowing traffic to flow from the pod’s eth0 to the host’s network stack.

# brctl show
bridge name     bridge id               STP enabled     interfaces
cni0            8000.a656432b14cf       no              veth1f7db117
...
docker0         8000.024216a031b6       no

Wireshark captures confirm that the VXLAN packet is received on the host’s eth0, decapsulated by the kernel, and handed to flannel.1, which then forwards it to the appropriate pod.