How Kubernetes Orchestrates Containers: From Architecture to Service Discovery

Kubernetes Architecture

Kubernetes has become the king of container orchestration, a cluster orchestration engine based on containers that offers cluster scaling, rolling upgrades and rollbacks, elastic scaling, self‑healing, service discovery, and many other capabilities.

This article quickly introduces Kubernetes and explains what is meant when we talk about Kubernetes.

From a macro perspective, Kubernetes consists of a Master, Nodes, and etcd.

The Master (control plane) manages the entire cluster and includes components such as API Server, Scheduler, and Controller, all of which interact with etcd for data storage.

API Server : Provides a unified entry point for resource operations, abstracting direct interaction with etcd and handling security, registration, and discovery.

Scheduler : Assigns Pods to Nodes according to scheduling rules.

Controller : Acts as the resource control center to ensure resources reach their desired state.

A Node is a worker that supplies compute power where containers actually run; it includes the container runtime, kubelet , and kube-proxy .

kubelet : Manages the lifecycle of containers, monitors them via cAdvisor, performs health checks, and periodically reports node status.

kube-proxy : Implements service discovery and load balancing inside the cluster by watching Service/Endpoint changes and updating load‑balancing rules.

Starting from Creating a Deployment

A Deployment is a controller resource used to orchestrate Pods; it will be introduced in detail later. Using a Deployment as an example, we can see what each component does during the creation of a Deployment resource.

kubectl sends a request to create a Deployment.

The API Server receives the request and writes the relevant resources to etcd; subsequent interactions between components and the API Server/etcd follow the same pattern.

The Deployment controller watches resource changes and issues a request to create a ReplicaSet.

The ReplicaSet controller watches resource changes and issues a request to create Pods.

The Scheduler detects unbound Pods and selects suitable Nodes through matching and filtering.

kubelet on the selected Node discovers the new Pod and creates it, then manages its lifecycle.

kube-proxy initializes Service‑related resources, including service discovery and load‑balancing network rules.

Through the coordinated work of these components, the process from a Deployment creation request to the normal operation of each Pod is completed.

Pod

Among Kubernetes' many API resources, a Pod is the most fundamental and smallest deployment unit.

Pods are designed as a container design pattern for tightly coupled containers that need to share network, storage, and configuration, such as a servlet container and a log collector.

Different containers in a Pod share the same network namespace via an infra container and can share storage by mounting the same volume, which corresponds to a directory on the host.

Container Orchestration

Container orchestration is Kubernetes' core strength. It provides many orchestration resources, such as Deployment for stateless applications, StatefulSet for stateful applications, DaemonSet for daemon processes, and Job/CronJob for batch workloads.

Using the widely adopted Deployment as an example, the relationship between Deployment, ReplicaSet, and Pod is hierarchical: a ReplicaSet controls the number of Pods, while a Deployment controls the version attribute of the ReplicaSet. This design enables two fundamental orchestration actions: horizontal scaling (adjusting replica count) and version control (updates/rollbacks).

Horizontal Scaling

Horizontal scaling is straightforward: modify the replica count controlled by the ReplicaSet, e.g., change from 2 to 3 to scale out, or decrease to scale in.

Update/Rollback

Updates and rollbacks illustrate the purpose of the ReplicaSet object. For example, to change a Deployment from version v1 with 2 replicas to version v2, the v1 ReplicaSet gradually reduces its Pods from 2 to 0 while the v2 ReplicaSet increases its Pods from 0 to 2. When only the v2 ReplicaSet remains, the update is complete; rollback performs the reverse.

Rolling Update

During a rolling update, Pods are upgraded one by one, ensuring that at least two Pods remain available while up to four Pods may serve traffic. This guarantees service continuity; if the new version has a bug, the remaining Pods keep the service alive, and a quick rollback is possible.

In practice, the RollingUpdateStrategy can be configured with maxSurge (how many new Pods the Deployment controller may create) and maxUnavailable (how many old Pods may be deleted).

Networking in Kubernetes

Having understood how container orchestration works, we now examine how containers communicate.

Node can communicate with its Pods.

Pods on the same Node can communicate with each other.

Pods on different Nodes can communicate with each other.

In simple terms, Pods on the same Node communicate via the cni0/docker0 bridge; Nodes reach Pods through the same bridge.

Cross‑Node Pod communication can be implemented with solutions like Flannel (VXLAN or host‑gateway mode). Flannel obtains network information of other Nodes from etcd, creates routing tables on the local Node, and enables inter‑Node communication.

Microservices – Service

Before proceeding further, we need to understand an important resource object: Service.

Why do we need a Service? In microservices, a Pod represents an instance, while a Service represents the microservice itself. Service solves two problems:

Pod IPs are not stable; a Service provides a stable endpoint.

Service performs load balancing across multiple Pods.

Service uses a label selector to pick appropriate Pods and builds an Endpoints list (the load‑balancing list). Typically, all Pods of the same microservice share a label such as app=xxx, and the Service uses the same selector app=xxx.

Service Discovery and Network Calls in Kubernetes

With the three‑way connectivity in place, we can explore how service discovery and network calls work in a microservice architecture on Kubernetes.

Service‑to‑Service Calls

East‑west traffic (service‑to‑service) mainly uses two modes: ClusterIP and DNS.

ClusterIP is a Service type where kube‑proxy creates a virtual IP (VIP) using iptables or IPVS. Accessing this VIP load‑balances traffic to the backing Pods.

DNS mode is straightforward: a ClusterIP Service has an A record like service-name.namespace-name.svc.cluster.local pointing to the ClusterIP address, so callers can simply use the Service name.

External Access

North‑south traffic (external requests entering the cluster) is handled by three common Service types: NodePort, LoadBalancer, and Ingress.

NodePort assigns a specific port on the host node; accessing that port forwards traffic to the Service.

LoadBalancer creates an external load balancer provided by a public cloud to expose the Service.

Ingress acts as a unified entry point, routing external requests to different Services based on rules; it typically combines NodePort and LoadBalancer under the hood.

In summary, we have covered the core concepts of Kubernetes, how it operates, and how microservices run on it, enabling us to understand what people are discussing when they talk about Kubernetes.