Automating Service Discovery and Load Balancing with Consul, HAProxy, and Docker in a Microservices Architecture

The article introduces micro‑service architecture, emphasizing the need to split a single application into small, independently deployable services that communicate via lightweight mechanisms such as load balancing, service discovery, and containerization.

It then describes the challenges faced in a non‑cloud environment, including resource contention, manual deployment steps, static configuration causing costly service inter‑dependencies, and difficulty in handling service changes.

To address these issues, the author proposes a new approach: containerized deployment using Docker, dynamic service discovery with Consul, and HAProxy as a unified external entry point that resolves service names via Consul’s DNS.

Consul Overview – Consul provides multi‑data‑center support, service discovery, health checks, key/value storage, and runtime orchestration. It enables services to register themselves and allows other services to discover them automatically.

HAProxy Configuration – HAProxy is configured with a custom resolvers consuldns section to perform dynamic DNS lookups against Consul, and frontend/backend definitions use ACLs to route traffic based on hostnames. Example resolver configuration:

resolvers consuldns
    nameserver dns1 127.0.0.1:53
    resolve_retries 200
    timeout retry 1s
    hold valid 10s

Frontend example for routing:

frontend serverA
    balance leastconn
    cookie JSESSIONID prefix
    bind 0.0.0.0:1000 accept-proxy
    capture request header Host len 128
    option httplog
    log-format %si:%sp %ci %ft %hrl %r %ST %B %Tt
    acl host_hostname1 hdr_dom(host) -i a.test.com
    acl host_hostname2 hdr_dom(host) -i b.test.com
    use_backend hostname1 if host_hostname1
    use_backend hostname2 if host_hostname2

Backend definitions reference the Consul resolvers and use service domain names:

backend hostname1
    server hostname1 a.service.consul:1000 resolvers consuldns maxconn 50000 check inter 2000 rise 2 fall 100

backend hostname2
    server hostname2 b.service.consul:1000 resolvers consuldns maxconn 50000 check inter 2000 rise 2 fall 100

Consul’s KV watch feature and consul-template are used to automatically update HAProxy configuration when services are added or removed, triggering a reload via a custom script.

Service registration example (JSON) and DNS lookup output illustrate how new services become discoverable instantly:

#web_service.json
{
    "service": {
        "name": "web",
        "port": 80,
        "id": "web",
        "address": "10.1.1.1",
        "check": {
            "id": "web",
            "name": "tcp",
            "tcp": "10.1.1.1:80",
            "interval": "60s",
            "timeout": "30s"
        }
    }
}

# nslookup web.service.consul
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    web.service.consul
Address: 10.1.1.1
Name:    web.service.consul
Address: 10.1.1.2
Name:    web.service.consul
Address: 10.1.1.3

The final summary highlights that combining HAProxy, Consul, and Docker yields high scalability, stability, and near‑full automation of the entire service lifecycle—from provisioning and configuration changes to deployment, scaling, and fault recovery—while eliminating the majority of manual intervention.