How Alibaba Cloud K8s Builds Its Cluster Network with Flannel – A Deep Dive

Overview

Alibaba Cloud K8s offers two network solutions—flannel and the calico‑based terway. The article focuses on the flannel implementation (version 1.12.6) and analyzes how the cluster network is built and how communication works.

Network Architecture

After configuration, the cluster consists of a cluster CIDR, VPC route tables, node networks, per‑node podCIDR, a virtual bridge cni0, and veth pairs linking pods to the bridge.

Three Configuration Levels

The network settings can be understood in three layers: cluster configuration, node configuration, and pod configuration. These correspond to three successive IP allocations: the cluster CIDR, a podCIDR subnet for each node, and an individual IP for each pod within its podCIDR.

Initial Stage – VPC and ECS

A VPC with CIDR 192.168.0.0/16 is created, and several ECS instances obtain IPs from this range.

Cluster Stage

The cluster creation console supplies a cluster CIDR to the node provisioning script, which passes it to kubeadm. kubeadm writes the CIDR into kube-controller-manager.yaml, enabling the controller to allocate a podCIDR for each node (e.g., Node A = 172.16.0.128/25, Node B = 172.16.8.1/25).

Node Stage

Two key actions occur:

The Cloud Controller Manager adds a VPC route entry for each node’s podCIDR, directing traffic destined for that subnet to the corresponding ECS.

A virtual bridge cni0 and its routing rules are created so that packets whose destination is a podCIDR are forwarded into the bridge’s LAN.

Pod Stage

The flanneld daemonset builds the network backbone, while the flannel CNI plugin creates a network namespace and a veth pair for each pod. One end of the veth joins cni0, and the other receives an IP from the pod’s subnet, completing the pod’s connectivity.

Note that flanneld (the daemonset) and the flannel CNI plugin are separate components: the former establishes the overlay network, the latter configures individual pod interfaces.

Communication Types

With the network in place, pods can communicate in four ways:

Local communication : containers within the same pod share a network stack and use the loopback interface.

Same‑node pod communication : occurs inside the cni0 bridge, behaving like a Layer‑2 LAN.

Cross‑node pod communication : packets travel from the source pod to cni0, out through the node’s eth0 to the VPC router, which forwards them to the destination node’s cni0 and finally to the target pod.

Pod‑to‑external communication : the node’s iptables SNAT rule (enabled by --ip-masq in flanneld) translates pod source addresses for traffic leaving the VPC.

Conclusion

The article breaks down Alibaba Cloud K8s networking into four stages—initial, cluster, node, and pod—clarifying how complex configurations map to logical IP allocations and how the combination of flanneld, the CNI plugin, and VPC routing enables reliable pod communication across the cluster.