Master Service Discovery with Consul: Deployment, Management & Scaling Guide

Application Background

HiAR is a next‑generation AR development platform that provides simple, powerful, cross‑platform AR services. Its cloud services rely on high availability, scalability, and service discovery.

Pain Points Without Service Discovery

Manual modification of Nginx/LVS configuration files and DNS records when adding nodes.

Manual configuration changes for service version releases.

Only semi‑automatic scripts, not fully automated.

DNS failures require maintenance.

Lack of service registration limits Docker usage.

With service discovery, node changes automatically update Nginx/LVS, DNS is unnecessary, and Mesos+Docker enable elastic scaling.

Why Choose Consul

Consul offers low operational cost, simple deployment, and a complete feature set suitable for small‑to‑medium teams.

Key features:

Service registration via HTTP API or DNS.

Service discovery via HTTP API or DNS.

Health checks (HTTP, TCP, Docker, custom scripts).

Configuration templates that auto‑update config files.

Additional features such as web UI and multi‑datacenter support.

Practical Experience

Deployment

Consul clusters consist of Server and Client roles. Typically 3‑5 servers are recommended.

Consul is a single binary; download the agent and place it on each node.

Example environment:

实验环境：
    server01 192.168.1.11
    server02 192.168.1.12
    server03 192.168.1.13
    client01 192.168.1.21

Start agents on the three servers:

[worker@server01 ~]$ consul agent -server -bootstrap-expect 2 -data-dir /tmp/consul -bind=192.168.1.11 -node=server01

[worker@server02 ~]$ consul agent -server -bootstrap-expect 2 -data-dir /tmp/consul -bind=192.168.1.12 -node=server02

[worker@server03 ~]$ consul agent -server -bootstrap-expect 2 -data-dir /tmp/consul -bind=192.168.1.13 -node=server03

Join the servers into a cluster:

[worker@server03 ~]$ consul join 192.168.1.11 192.168.1.12

Start a client node and join the server cluster:

[worker@client01 ~]$ consul agent -data-dir /tmp/consul -bind=192.168.1.21 -node=client01

[worker@client01 ~]$ consul join 192.168.1.11

[worker@client01 ~]$ consul agent -data-dir /tmp/consul -bind=192.168.1.21 -node=client01 -rejoin

Application

Consul works together with Nginx, LVS, Docker, etc., to provide elastic scaling. Nginx runs Consul Agent and Consul Template; the template periodically queries the local agent and updates Nginx configuration.

$ consul-template -consul 127.0.0.1:8500 -template "/etc/nginx/conf/vhosts/test.ctmpl:/etc/nginx/conf/vhosts/test.conf:nginx -s reload"

Template snippet for upstream configuration:

upstream test-cluster {
  ip_hash;{{range service "test"}}
  server {{.Address}}:{{.Port}};{{end}}
}

Service registration can be done via a JSON file (consul.json) with health checks, then started with the agent:

{
  "service": {
    "name": "test",
    "port": 9999,
    "check": {
      "tcp": "127.0.0.1:9999",
      "interval": "10s"
    }
  }
}

$ consul agent -data-dir /tmp/consul -node=test -bind=192.168.1.21 -config-dir=/tmp/consul.json

Management

Node processes are managed with systemd (or supervisord/upstart). Consul also provides a web UI to view services, nodes, health checks, and status.

Upgrade

Upgrade strategies depend on version specifics, compatibility, or standard procedures. Recommended order: upgrade follower servers first, then the leader, and finally all client nodes.

Conclusion

Introducing service registration and discovery transforms system architecture, enabling load balancing, health checks, heartbeat monitoring, elastic scaling, high availability, and smooth gray‑release deployments.