Deep Dive into Kubernetes Networking: Layers, Models, and Plugins

Kubernetes Networking Deep Dive

After the previous learning we know Kubernetes consists of various components and objects; this section addresses how it solves the following networking questions: Docker container‑to‑Docker container networking Pod‑to‑Pod networking Pod‑to‑Service networking Internet‑to‑Service networking

1 Basic Introduction

1.1 What is Kubernetes Network

Kubernetes network refers to the network architecture that enables communication between different components in a Kubernetes cluster. In Kubernetes each container gets its own IP address; containers form Pods, which are the smallest scheduling unit. The design goal is to support multiple network models and provide pluggable network plugins, allowing Kubernetes to run in various cloud and physical environments.

Kubernetes network is usually divided into four layers: Container Network Interface (CNI) Pod network Service network Ingress network

Container Network Interface (CNI) Layer

The underlying layer is CNI, an independent plugin system that assigns IP addresses to containers, creates network interfaces and configures the network environment. CNI plugins can be used in various cloud and physical environments such as AWS, GCP, Azure, OpenStack, Bare metal.

This layer solves Docker‑to‑Docker container networking.

Pod Network Layer

Pod network provides each Pod with a unique IP address and network namespace. Containers in a Pod can communicate via localhost (127.0.0.1) without external networking, achieving low latency and high throughput.

Within a Pod, containers share the Pod’s IP address and network namespace; they can use localhost or the Pod IP for communication. Port mapping can be defined in the Pod spec.

Example: two containers A and B in a Pod, A sends an HTTP request to B using localhost and B’s port; B returns an HTTP response.

Service Network Layer

The Service layer defines network communication between Services, providing a virtual IP that routes requests to backend Pods. Service network can use types such as ClusterIP, NodePort, LoadBalancer.

This layer solves Pod‑to‑Service networking.

Ingress Network Layer

Ingress is the top layer that allows external traffic into the Kubernetes cluster and routes requests to different Services. Ingress can be implemented with controllers like Nginx, Traefik, HAProxy.

This layer solves Internet‑to‑Service networking.

2 Kubernetes Network Models

Kubernetes network model provides a framework for container network connectivity inside and outside the cluster. It includes: Pod‑to‑Pod communication using CNI plugins (Flannel, Calico, Weave Net, etc.). Pod‑to‑Service communication via Service IP or DNS name. Pod‑to‑Node communication via node IP or localhost. Service exposure to external networks via Ingress, NodePort, or LoadBalancer. The model offers flexible, scalable, highly available and secure networking for containerized applications.

Network Solutions

Container‑to‑container communication without port mapping or NAT. Service discovery via virtual IP. Load balancing (Round Robin, IP Hash, Least Connection). Network isolation via policies. External traffic management via Ingress.

Kubernetes Network Plugins

Kubernetes provides multiple network plugins to enable container‑to‑container and container‑to‑external communication. Kube‑router Flannel Calico Weave Net Cilium

1 Kube‑router

Kube‑router is a BGP‑based container network solution that creates a virtual network and manages container communication using BGP. It assigns a unique IP to each container and adds these IPs to the routing table. Supports flat, mesh, and point‑to‑point topologies.

Usage Example

Example YAML to create a Deployment, Service, and a Pod that uses Kube‑router.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
    - name: http
      port: 80
      targetPort: 80
  type: ClusterIP
---
apiVersion: v1
kind: Pod
metadata:
  name: kube-router-pod
spec:
  containers:
    - name: kube-router-container
      image: kube-router/kube-router:v1.3
      command:
        - kube-router
        - run
      args:
        - --run-router=false
        - --run-firewall=false
        - --run-service-proxy=false
        - --run-egress=false
        - --enable-cni=true
        - --cni-bin-dir=/opt/cni/bin
        - --cni-conf-dir=/etc/cni/net.d
        - --cni-network-config='{
            "cniVersion": "0.3.1",
            "name": "kube-router",
            "type": "kube-router"
          }'
  volumeMounts:
    - name: cni-bin
      mountPath: /opt/cni/bin
    - name: cni-conf
      mountPath: /etc/cni/net.d
  volumes:
    - name: cni-bin
      hostPath:
        path: /opt/cni/bin
    - name: cni-conf
      hostPath:
        path: /etc/cni/net.d

2 Flannel

Flannel is a VXLAN or UDP‑based virtual network that creates a virtual network on each node to enable container communication. It allocates a unique IP range per node and maps container IPs into that range. Flannel relies on etcd or another distributed key‑value store for network configuration.

Usage Example

Example YAML to create a Deployment, Service, and a Pod that uses Flannel.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
    - name: http
      port: 80
      targetPort: 80
  type: ClusterIP
---
apiVersion: v1
kind: Pod
metadata:
  name: flannel-pod
spec:
  containers:
    - name: flannel-container
      image: quay.io/coreos/flannel:v0.14.0
      command:
        - /opt/bin/flanneld
      args:
        - --ip-masq
        - --kube-subnet-mgr
        - --iface=eth0
      securityContext:
        privileged: true
      volumeMounts:
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
  volumes:
    - name: flannel-cfg
      configMap:
        name: kube-flannel-cfg

3 Calico

Calico is a BGP‑based container network solution that assigns unique IPs to containers and uses BGP to add these IPs to routing tables, also providing strong network security mechanisms.

4 Weave Net

Weave Net is a VXLAN/UDP‑based virtual network that creates a virtual network across the cluster, assigning unique IPs to containers and supporting various topologies.

5 Cilium

Cilium leverages eBPF to intercept and manage container communication at the kernel level, supporting multiple network and application layer protocols and offering robust security.

Conclusion

Kubernetes offers a variety of network plugins that can be chosen based on specific network requirements and environments; when selecting and deploying a plugin, considerations such as reliability, performance and security are essential.