Mastering Kubernetes Networking: From Containers to Services

Kubernetes is built for running distributed clusters, making networking a core component; understanding its network model helps you run, monitor, and troubleshoot applications correctly.

The network is complex with many concepts such as network namespaces, virtual interfaces, IP forwarding, and NAT, which can be challenging for newcomers.

Kubernetes imposes the following requirements on any network implementation:

All Pods can communicate with all other Pods without NAT.

All nodes can communicate with all Pods without NAT.

A Pod's IP is seen identically by other Pods.

Given these constraints, we need to address several networking problems:

Container‑to‑container networking

Pod‑to‑Pod networking

Pod‑to‑Service networking

Internet‑to‑Service networking

We will discuss these issues and their solutions.

Container‑to‑Container Networking

Typically, VM networking is viewed as direct interaction with Ethernet devices (see Figure 1).

In Linux, each running process lives in a network namespace that provides its own routing, firewall rules, and network devices, effectively giving each process a separate network stack.

Linux users can create a network namespace with the ip command, e.g.: $ ip netns add ns1 After creation, a mount point appears under /var/run/netns, preserving the namespace even without attached processes.

You can list the mount points with:

$ ls /var/run/netns
ns1
$ ip netns
ns1

By default, Linux assigns each process to the root network namespace to provide external access (see Figure 2).

In Docker, a Pod consists of multiple containers sharing a network namespace, giving them the same IP address and port space. Docker adds the containers to the namespace using the -net=container:sandbox-container flag (see Figure 3).

Pods can also share volumes defined as part of the Pod specification.

Pod‑to‑Pod Networking

Each Pod has a real IP address and uses it to communicate with other Pods. For Pods on the same node, communication relies on Linux virtual Ethernet (veth) pairs and a bridge.

A veth pair connects the root network namespace to the Pod's namespace; one end is placed in the root namespace, the other in the Pod's namespace (see Figure 4).

Pods are connected to the node's root namespace via a Linux bridge, which functions like a virtual switch, learning MAC addresses and forwarding frames based on a forwarding table.

Same‑Node Pod Communication

Each Pod's network namespace is linked to the root namespace via veth devices and a bridge, enabling traffic between Pods on the same node (see Figure 6).

When pod1 sends a packet from its eth0, it travels through veth0(1) to the bridge cbr0, which forwards it to veth1(3), and finally to pod2's eth0(4). Throughout, each Pod communicates via its own

localhost

eth0

, and the bridge uses ARP to resolve MAC addresses.

Kubernetes requires Pods to be reachable across nodes using their IPs, so cross‑node communication is also needed.

Cross‑Node Pod Communication

Each node is allocated a CIDR block for the Pods it runs. When traffic destined for a Pod's CIDR arrives at a node, the node forwards it to the correct Pod (see Figure 7).

In this scenario, a packet from pod1 on node A traverses its veth pair to the root namespace, reaches the bridge, fails ARP, and is sent to the default route ( eth0), exiting the node. The external network routes the packet to node B based on the CIDR, where it re‑enters the root namespace, is bridged to the appropriate veth, and finally reaches the target Pod.

AWS provides a CNI plugin that creates an ENI for each Pod, allowing Pod IPs to be routable within the VPC. The plugin runs as a DaemonSet, and the kubelet requests an IP from the ENI pool, attaching it to the Pod via virtual devices and a bridge, enabling cross‑node Pod traffic.

Pod‑to‑Service

Pod IPs can change due to scaling or node restarts. Kubernetes uses a Service object to provide a stable virtual IP (VIP) that abstracts a set of Pods.

A Service assigns a clusterIP (VIP) and load‑balances traffic to the backing Pods. The kube‑proxy configures iptables (or IPVS) rules to rewrite traffic from the Service VIP to a selected Pod IP.

netfilter and iptables

Kubernetes relies on Linux's netfilter framework for packet filtering and NAT. The iptables user‑space tool defines tables of rules that kube‑proxy populates to route Service traffic.

When a packet matches a Service VIP, iptables rewrites the destination IP to a chosen Pod IP. On the return path, iptables rewrites the source IP back to the Service VIP so the client sees a consistent address.

IPVS

Newer Kubernetes versions can use IPVS, built on netfilter, providing kernel‑level load balancing with hash tables for higher scalability.

When kube‑proxy runs in IPVS mode, it creates a virtual IPVS interface, binds the Service VIP, and adds IPVS servers for each Pod.

Pod‑to‑Service Communication

A packet leaves the Pod via eth0, reaches the bridge, then the node's default route ( eth0) sends it outward. iptables intercepts the packet, rewrites the destination from the Service VIP to a specific Pod IP, and conntrack records the mapping for return traffic.

Service‑to‑Pod Communication

On the return path, the Pod's response is captured by iptables, which rewrites the source IP back to the Service VIP using conntrack, then the packet traverses the bridge back to the Pod's namespace.

External‑to‑Service Communication

Exposing services externally involves two main challenges: routing traffic from a Service to the Internet and routing inbound Internet traffic to the Service.

Outbound Traffic

In AWS VPC, each node has a private IP. An Internet‑bound gateway performs NAT for VM IPs, but not for Pod IPs. Kubernetes uses iptables to perform source NAT, making packets appear to originate from the VM, allowing the gateway to translate them to a public IP (see Figure 10).

Inbound Traffic

Ingress can be handled via a Service LoadBalancer or an Ingress controller.

LoadBalancer

When a Service is created with type LoadBalancer, the cloud provider provisions a load balancer that exposes an external IP. Traffic from the load balancer is distributed across nodes, and iptables on each node routes it to the appropriate Pod.

LoadBalancer to Service

Incoming packets arrive at the load balancer, which forwards them to a node; iptables on the node rewrites the destination to the selected Pod, and conntrack ensures correct return path rewriting.

Ingress Controller

Ingress operates at the HTTP/HTTPS layer, built on top of Services. A NodePort Service exposes a port on each node; the Ingress controller maps HTTP paths to Services.

In AWS, the ALB Ingress controller creates an Application Load Balancer, TargetGroups for each Service, and listeners that route traffic based on path rules (see Figure 12).

Ingress traffic follows a similar lifecycle to LoadBalancer traffic, but the controller knows URL paths and can route accordingly.

Conclusion

This article introduced the Kubernetes networking model and common networking tasks. The topic is broad and deep; consider this article a starting point for deeper exploration of areas that interest you.

Source: https://sookocheff.com/post/kubernetes/understanding-kubernetes-networking-model (© original author)