Demystifying Kubernetes Service Discovery: From Basics to Deep Dive

K8S Network Basics

Before exploring service discovery, understand that Kubernetes workloads run in containers inside Pods, each Pod has a unique routable IP within a flat overlay network (often VXLAN).

These three factors let applications communicate directly without NAT.

Dynamic Network

When scaling horizontally, new Pods receive new IPs; when scaling down, old Pods and their IPs are removed, making network state appear chaotic.

Without a central solution, each application would need to monitor Pod health and manage its own list, which is inefficient.

Service Brings Stability

Kubernetes Service objects create a stable network endpoint in front of a group of Pods and perform load balancing.

Clients talk to the Service, which forwards traffic to the appropriate Pods.

The Service’s name, IP, and port never change, even as underlying Pods are added, removed, or updated.

K8S Service Anatomy

A Service consists of:

Front‑end: immutable name, IP, and port.

Back‑end: a set of Pods selected by label selectors.

The front‑end’s stability avoids DNS cache issues for clients.

The back‑end is dynamic; Pods matching the selector receive traffic via simple layer‑4 round‑robin load balancing.

Deep Dive into K8S Service Discovery

Service discovery comprises two functions: registration and lookup.

Service Registration

Kubernetes uses DNS as the service registry. Each cluster runs a DNS service (CoreDNS) in the kube-system namespace. Every Service automatically registers its name and ClusterIP with this DNS.

Registration steps:

POST a new Service definition to the API Server.

Request passes authentication, authorization, and admission checks.

Service receives a ClusterIP (virtual IP) stored in the cluster data store.

Configuration propagates across the cluster.

CoreDNS creates the corresponding DNS A record.

CoreDNS watches the API Server for new Service objects and creates DNS entries mapping Service names to ClusterIPs.

Endpoint Object

After the Service front‑end registers, the back‑end consists of a list of Pods. The Service’s label selector determines which Pods belong to this list, stored in an Endpoints object.

Service Discovery

When a Pod wants to reach another service, it queries the cluster DNS for the Service name, receives the ClusterIP, and sends traffic to that virtual IP.

Because the Service network has no direct routing, packets are first sent to the node’s default bridge (cbr0), then to the node’s network stack.

Each node runs kube-proxy, which watches Service objects and creates iptables or IPVS rules that capture traffic destined for a Service’s ClusterIP and forward it to the IPs of healthy Pods listed in the Endpoints object.

iptables operates at layer 3/4, while IPVS (the newer default) provides more efficient layer‑4 load balancing.

Summary

Creating a Service allocates a virtual ClusterIP and registers the name and IP in cluster DNS; an Endpoints object tracks the current set of healthy Pods. All nodes configure iptables/IPVS rules so that traffic to the ClusterIP is rewritten to the appropriate Pod IP, enabling stable, load‑balanced communication between Pods.