How Do Containers Communicate in Kubernetes? A Deep Dive into Network Basics and CNI

Container Network Basics

In Kubernetes, network connectivity between containers is essential. Kubernetes does not implement its own container network but relies on plug‑in mechanisms. The basic principles are:

Pods on any node can communicate directly without NAT.

Nodes and Pods can communicate; Pods can reach any network.

Each Pod has its own network stack, and all containers in a Pod share it.

Linux Container Network Fundamentals

A Linux container’s network stack is isolated in its own network namespace, which includes a network interface, loopback device, routing table and iptables rules. The key Linux networking features used are:

Network namespaces – isolate protocol stacks.

Veth pairs – virtual Ethernet devices that connect namespaces.

Iptables/Netfilter – packet filtering and NAT.

Linux bridge – a layer‑2 virtual switch forwarding frames based on MAC addresses.

Routing – kernel routing table determines packet forwarding.

Containers on the same host communicate via the default docker0 bridge. A Veth pair connects each container to the bridge; one end appears as eth0 inside the container, the other as a veth… device on the host.

docker run -d --name c1 hub.pri.ibanyu.com/devops/alpine:v3.8 /bin/sh

Inspecting the container shows eth0 with IP 172.17.0.2 and the default route via that interface.

docker exec -it c1 /bin/sh
# ifconfig
# route -n

The host side of the Veth pair (e.g., veth20b3dac) is attached to docker0, as shown by brctl show.

# brctl show
docker0  8000.02426a4693d2  no  veth20b3dac

Two containers on the same host can ping each other because the ARP request is broadcast on docker0, the bridge forwards it to the appropriate Veth peer, and the reply reaches the source container.

Cross‑Host Network Communication

By default Docker containers on different hosts cannot reach each other by IP. Kubernetes introduces the CNI (Container Network Interface) API, allowing plug‑ins such as Flannel, Calico, Weave, and Contiv to provide cross‑host networking.

CNI plugins create a dedicated bridge (default name cni0) instead of docker0. The three CNI networking modes are:

Overlay – encapsulates pod traffic in tunnels (e.g., Flannel VXLAN, Calico IPIP).

Layer‑3 routing – uses routing tables without tunnels, requiring a shared L2 network (e.g., Flannel host‑gw, Calico BGP).

Underlay – pods and hosts are on the same L3 network; routing is handled directly.

Flannel in host‑gw mode installs a route on each node pointing to the pod CIDR of other nodes, e.g. 10.244.1.0/24 via 10.168.0.3 dev eth0 Calico uses BGP to distribute routes. Its components include the Calico CNI plugin, Felix (maintains host routes), BIRD (BGP daemon), and ConfD (configuration). Calico does not create a bridge; instead it adds a route for each pod’s Veth pair, for example: 10.92.77.163 dev cali93a8a799fe1 scope link In large clusters the node‑to‑node mesh BGP topology scales poorly, so a router‑reflector (RR) mode is recommended for clusters larger than ~50 nodes.

When nodes are not on the same L2 network, Calico can fall back to IPIP overlay mode, adding routes such as: 10.92.203.0/24 via 10.100.1.2 dev tunnel0 Alternatively, underlay routing can be used if the physical network supports BGP, allowing direct L3 communication without encapsulation.

In summary, Kubernetes networking can be implemented with simple host‑gw bridges for small, single‑LAN clusters, while Calico with BGP or IPIP is better suited for larger, multi‑datacenter deployments.