How Kubernetes Connects Containers: From Docker Bridge to CNI Plugins

Container Network Basics

In Kubernetes, container networking is essential for inter‑pod communication, and Kubernetes relies on plug‑in mechanisms rather than implementing its own network stack. The basic principles are:

Pods on any node can communicate directly without NAT.

Nodes and pods can talk to each other, and pods can access any network.

Each pod shares a single network namespace, so the address seen inside the pod matches the external view.

A Linux container’s network stack is isolated in its own network namespace, which includes a network interface, loopback device, routing table, and iptables rules.

Key Linux networking components used for container networking are:

Network Namespace : isolates independent network stacks.

Veth Pair : a pair of virtual Ethernet devices that connect different namespaces.

Iptables/Netfilter : provide packet filtering and NAT capabilities.

Bridge : a layer‑2 virtual device that forwards frames based on MAC addresses.

Routing : Linux routing tables determine packet forwarding paths.

On a single host, Docker creates a

docker0

bridge. Containers connect to this bridge via a veth pair; one end appears as

eth0

inside the container, the other end appears on the host (e.g.,

veth20b3dac

).

<code>docker run -d --name c1 hub.pri.ibanyu.com/devops/alpine:v3.8 /bin/sh</code>

<code>docker exec -it c1 /bin/sh</code><code>/ # ifconfig</code><code>eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:02</code><code>        inet addr:172.17.0.2  Bcast:172.17.255.255  Mask:255.255.0.0</code><code>/ # route -n</code><code>0.0.0.0         172.17.0.1      0.0.0.0         UG</code>

The host sees the other end of the veth pair (e.g.,

veth20b3dac

) attached to

docker0

. Using

brctl show

confirms the bridge association.

<code># brctl show</code><code>docker0        8000...   no   veth20b3dac</code>

Launching a second container and pinging its IP demonstrates that packets travel from the container’s

eth0

to the host bridge, then through the veth pair to the target container.

<code>docker run -d --name c2 -it hub.pri.ibanyu.com/devops/alpine:v3.8 /bin/sh</code><code>docker exec -it c1 /bin/sh</code><code>/ # ping 172.17.0.3</code>

The ping succeeds because the destination IP matches a direct route (gateway

0.0.0.0

) and the bridge forwards the packet at layer 2 after ARP resolution.

Cross‑Host Network Communication

Docker’s default setup cannot reach containers on different hosts. Kubernetes introduces the CNI (Container Network Interface) API, allowing plug‑ins such as Flannel, Calico, Weave, and Contiv to provide cross‑host networking.

CNI plugins typically create a dedicated bridge (e.g.,

cni0

) on each node. The three common CNI networking modes are:

Overlay : encapsulates the entire pod network over the underlying network using tunnels (e.g., VXLAN, IPIP). Plugins: Flannel (UDP/VXLAN), Calico (IPIP).

Layer‑3 Routing : relies on routing tables without tunnels, requiring the nodes to be on the same L2 network. Plugins: Flannel (host‑gw), Calico (BGP).

Underlay : uses the underlying L3 network directly, with no bridge, but depends on the physical network’s capabilities. Plugin example: Calico (BGP).

Example of Flannel host‑gw routing:

<code>10.244.1.0/24 via 10.168.0.3 dev eth0</code>

Calico’s architecture consists of four components: the Calico CNI plug‑in,

felix

(maintains host routing rules),

BIRD

(distributes routes via BGP), and

confd

(configuration management).

Calico does not create a bridge; instead, it assigns a veth pair to each pod and adds a host‑side route, e.g.:

<code>10.92.77.163 dev cali93a8a799fe1 scope link</code>

When nodes are not on the same L2 segment, Calico can fall back to IPIP overlay mode, adding a tunnel route such as:

<code>10.92.203.0/24 via 10.100.1.2 dev tunnel0</code>

In large clusters, Calico’s default node‑to‑node mesh can cause O(N²) BGP sessions; therefore, a Router‑Reflector (RR) topology is recommended for clusters larger than ~50 nodes.

Reference

https://github.com/coreos/flannel/blob/master/Documentation/backends.md

https://coreos.com/flannel/

https://docs.projectcalico.org/getting-started/kubernetes/

https://www.kancloud.cn/willseecloud/kubernetes-handbook/1321338

Source: http://tech.ipalfish.com/blog/2020/03/06/kubernetes_container_network/