Mastering Kubernetes Networking: From Core Model to Production‑Ready Practices

Core Network Model and CNI

Kubernetes requires each Pod to have a unique IP address and to communicate with other Pods without NAT. The IP‑per‑Pod model treats each Pod as an independent host, sharing the network namespace for containers. Communication requirements include direct Pod‑to‑Pod traffic across nodes, Pod‑to‑Service via stable ClusterIP, and External‑to‑Service for external clients.

Pod Networking : IP‑per‑Pod model, containers share the Pod’s network namespace.

Communication : Pod‑to‑Pod (cross‑node direct), Pod‑to‑Service (ClusterIP), External‑to‑Service (external client access).

Container Network Interface (CNI)

Kubernetes itself does not implement networking; it delegates to CNI plugins. The typical workflow is:

Kubelet creates a pause container and a network namespace.

Kubelet invokes the CNI plugin’s ADD command.

The plugin allocates an IP, configures a veth pair, and sets routing.

The result is returned to Kubelet.

Common CNI plugins include:

Flannel (simple VXLAN/host‑gw overlay)

Calico (BGP‑based, supports NetworkPolicy, high performance)

Cilium (eBPF‑based, most feature‑rich)

Weave Net (simple overlay)

Supplement: Large clusters can enable IPv4/IPv6 dual‑stack mode, requiring support from both CNI and kube‑proxy.

Service Networking

Service types:

ClusterIP : virtual IP inside the cluster.

NodePort : exposes <NodeIP>:<NodePort>.

LoadBalancer : relies on cloud provider LB.

ExternalName : maps to an external DNS name.

Implementation via kube‑proxy watches Service/EndpointSlice changes and programs forwarding rules using one of:

iptables : default, mature but can degrade with many rules.

IPVS : kernel L4 load balancer, high performance.

userspace : deprecated.

Ingress and API Gateways

Ingress : manages external HTTP/HTTPS entry, supports host/path routing and TLS.

Ingress Controllers (required): Nginx, Traefik, HAProxy, Istio Gateway.

API Gateway mode : for authentication, rate‑limiting, circuit‑breaking, using Kong, Tyk, Gloo, or service‑mesh gateways.

NetworkPolicy

By default all Pods can communicate; NetworkPolicy adds label‑based firewall capabilities.

Pod Selector : selects target Pods.

Policy Types : Ingress / Egress.

Supported CNI : Calico, Cilium, Weave Net (Flannel does not support).

Example allowing only the frontend Pods to reach api‑server:8080:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-allow-only-frontend
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api-server
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080

DNS and Service Discovery

CoreDNS provides service name resolution.

Service DNS format: {service}.{namespace}.svc.cluster.local → ClusterIP.

Pod DNS is managed by StatefulSet or hostname/subdomain settings.

Service Mesh and eBPF

When micro‑service count grows, a service mesh offers stronger traffic governance.

Architecture : data plane (sidecar Envoy) + control plane (central traffic management).

Features : canary/blue‑green deployments, traffic mirroring, mTLS, zero‑trust access, metrics, logging, distributed tracing.

Implementations : Istio, Linkerd, Consul Connect.

eBPF‑based solutions:

Cilium : eBPF, no sidecar, superior performance.

Provides L3/L4/L7 policies, traffic observation, transparent encryption.

Advanced CNI Features and Performance Optimizations

IPAM management : allocate separate IP pools per namespace/node pool to avoid exhaustion.

MTU configuration : overlay encapsulation requires uniform MTU to prevent large‑packet loss.

SR‑IOV / DPDK : direct Pod‑to‑physical NIC for high‑performance scenarios.

kube‑proxyless Mode

Problem : iptables/IPVS become bottlenecks at massive scale.

Alternatives : Cilium kube‑proxy replacement (eBPF) and Katran (Facebook open‑source eBPF L4 load balancer).

Pod External Network Access

Default Pods can egress; can restrict with Egress NetworkPolicy or an Egress Gateway.

SNAT typically rewrites Pod egress to Node IP; to preserve Pod IP, use Calico BGP or Cilium Egress Gateway.

Multi‑Cluster and Multi‑Tenant Networking

Cross‑cluster solutions:

Submariner (overlay, works with many CNI).

Cilium Cluster Mesh (eBPF, no tunnel).

Istio multi‑cluster (federated control plane).

Multi‑tenant isolation uses namespaces + NetworkPolicy; for stricter isolation, virtual clusters (vCluster) can be employed.

Network Security and Compliance

Zero‑trust networking: combine NetworkPolicy with mTLS for fine‑grained access control.

Compliance (e.g., financial/GDPR) requires full traffic encryption, audit logging, and network isolation reports.

Observability and Troubleshooting

Cilium Hubble – visualizes network traffic.

Istio Kiali – service‑mesh topology.

Weave Scope – network topology view.

Key Metrics

Pod RTT latency.

Service forwarding success rate.

DNS resolution success rate.

Network packet loss rate.

Common Commands

Pod network issues: kubectl describe pod, kubectl exec -it <pod> -- ping <ip>, curl -v <url>.

Service access problems: kubectl get svc, kubectl get endpointslices, curl <cluster-ip>:<port>.

DNS problems: kubectl get pods -n kube-system nslookup <service> (check /etc/resolv.conf).

Node anomalies: ip addr, ip route, iptables -t nat -L -n, tcpdump -i any port 53.

CNI failures: view plugin pod logs or journalctl -u kubelet -f.

Conclusion

A layered understanding of Kubernetes networking includes:

Bottom layer – CNI plugins provide Pod‑to‑Pod connectivity.

Service layer – kube‑proxy and Service objects deliver load balancing and discovery.

Ingress layer – Ingress Controllers or API gateways handle external traffic.

Security layer – NetworkPolicy enforces communication rules.

Advanced layer – Service mesh or eBPF‑based solutions add powerful traffic governance and observability.

Choosing the right combination of CNI, Ingress Controller, and NetworkPolicy or service‑mesh yields a high‑performance, secure, and observable Kubernetes network for production environments.