Understanding Kubernetes Load Balancing: Internal and External Strategies

Kubernetes Overview

Kubernetes (K8s) is the most widely used container orchestration platform, designed to handle large‑scale container scheduling and automated deployment. It provides resource scheduling, service discovery, auto‑scaling, rolling upgrades, and self‑healing capabilities.

Core Functionality: Service Discovery + Load Balancing

Service discovery combined with load balancing is the most frequently used feature of Kubernetes.

Internal Load Balancing

Inside the cluster, load balancing is primarily achieved through Service objects, especially ClusterIP , NodePort , and Headless Service . When a client accesses a Service, kube-proxy runs on each Node and forwards incoming traffic to the backend Pods. kube-proxy supports multiple modes, notably iptables and ipvs . The iptables mode uses kernel packet filtering and NAT rules to perform simple round‑robin or random forwarding.

External Load Balancing

Outside the cluster, load balancing is typically handled by an Ingress Controller or by cloud provider load balancers such as ELB or SLB.

Load Balancing Strategies

When using an Ingress Controller or a Service Mesh (e.g., Istio, Linkerd), the following algorithms are commonly employed:

Round Robin : Requests are sent to Pods in order, providing a simple and evenly distributed approach.

Least Connections : New connections are directed to the backend with the fewest active connections, useful when request latency varies significantly.

Consistent Hash / Ring Hash : A hash of a request key (e.g., user ID, IP, Cookie) maps the request to a specific Pod, ideal for cache clusters or scenarios requiring session affinity.

Weighted Round Robin / Weighted Least Connections : Pods are assigned weights (based on CPU, memory, or node performance); higher‑weight Pods receive more traffic, suitable for heterogeneous deployments.