How Docker Handles Service Discovery and Load Balancing: Strategies and Trade‑offs

Background

Docker is an open‑source container engine originally created in early 2013 as an internal project at dotCloud. After being open‑sourced it gained wide attention; the company later renamed itself Docker Inc. Docker is written in Go, licensed under Apache 2.0, and its source code is hosted on GitHub.

In simple terms Docker provides a process‑isolation model that allocates resources, aiming to deliver a lightweight operating‑system‑level virtualization solution.

Key Terminology

Dubbo : an Alibaba‑maintained high‑performance RPC framework that integrates with Spring.

LVS : Linux Virtual Server, a virtual server cluster system.

IPVS : IP Virtual Server, a load‑balancing technology built on top of LVS.

Nginx : a lightweight web server, reverse proxy, and mail proxy.

HAProxy : an open‑source software written in C that provides high availability and load balancing for TCP/HTTP.

North‑South traffic : traffic entering or leaving the container cluster.

East‑West traffic : traffic between containers inside the cluster.

Technical Challenges of Containerization

In the default network model a container’s IP address changes on every restart, making it difficult to guarantee stable IPs in large distributed systems.

Frequent IP changes prevent deterministic client‑to‑server endpoint discovery, raising the question: how can a client locate a server when container IPs are volatile?

Solution Overview

Service discovery can be classified according to whether the client is aware of the discovery mechanism.

Client‑side discovery

The client subscribes to a registry (e.g., Dubbo, DNS). The registry pushes a list of service endpoints to the client, allowing the client to select an active instance.

Server‑side discovery

The server publishes a fixed endpoint; the client always contacts this endpoint, which internally forwards the request to the appropriate container behind a load balancer. This approach works well for applications that are not built on a service‑oriented framework.

Server‑side discovery is effectively “load balancer + automatic route‑configuration updates”.

Micro‑service Discovery Options

DNS

Pros: Docker ≥ 1.10 natively supports intra‑cluster DNS‑based service discovery.

Cons: DNS TTL can cause stale records; even with TTL = 0 some resolvers cache results.

Kernel‑space LVS/IPVS

Pros: Implemented in the kernel, provides Layer‑4 load balancing without copying packet payloads, offering high efficiency.

Cons: Lacks Layer‑7 features; each service consumes a host port, which can lead to port conflicts.

User‑space Nginx

Pros: Supports both Layer‑4 and Layer‑7 load balancing, multi‑process model leverages multi‑core CPUs, includes caching and can serve static files. Layer‑7 routing can differentiate hosts, paths, and perform redirects.

Cons: Limited scheduling algorithms and health‑check strategies.

User‑space HAProxy

Pros: Pure software load balancer supporting many scheduling algorithms (round‑robin, least‑conn, source‑IP, etc.) and comprehensive health checks (TCP, HTTP, executable probes).

Cons: Cannot serve static files and lacks built‑in caching.

Container Service Load‑Balancing Architecture

Combining the above analyses, the following solutions are recommended.

Docker’s built‑in DNS service discovery (available from version 1.1) provides:

Independent DNS resolution per container.

Resolution by container name or alias across the network scope.

Link‑alias based DNS that avoids name conflicts.

Proxying of external DNS queries.

SLB (software load balancer) dynamically binds containers: a Swarm manager monitors container health and adds healthy containers to the SLB backend, removing unhealthy ones.

HAProxy can achieve dynamic service discovery by running a script inside the HAProxy container that regenerates the load‑balancing configuration based on container status and reloads HAProxy.

Rolling‑update without downtime is realized by updating containers in two batches (A and B) while always keeping at least one healthy instance serving traffic. The process involves removing batch A from the load balancer, updating it, re‑adding it, then repeating for batch B.

Gray‑release is achieved by adjusting SLB or HAProxy weights to route a portion of traffic to a new version while the rest continues to use the stable version.

Scenario‑Based Service Forms

Simple routing service : A wrapper around HAProxy provides dynamic discovery of running containers and exposes a public IP via an external SLB. This satisfies two needs:

Expose Layer‑7 service endpoints to the Internet.

Provide internal load balancing and service discovery within the cluster using Docker’s DNS resolver together with HAProxy health checks.

Typical request flow:

Docker DNS resolves restserver.local to the HAProxy container IP, preferring the HAProxy on the same node.

The client sends a request to restserver.local, which first reaches the HAProxy wrapper.

The HAProxy queries the discovery service for backend information and forwards the request to the appropriate RestServer container.

Summary of Routing Services by Scenario

North‑South traffic (external‑to‑cluster) benefits from an SLB that forwards Layer‑7 traffic to HAProxy containers, which then dispatch to backend services. East‑West traffic (container‑to‑container) uses Docker’s internal DNS resolver combined with HAProxy for intra‑cluster load balancing.