Using ETCD for Leader Election and High Availability: Architecture, Installation, and Go Implementation

ETCD is an open‑source, highly consistent distributed key‑value store used for configuration sharing and service discovery in systems such as Kubernetes, ROOK, CoreDNS, M3, and OpenStack.

The article introduces ETCD's main features, its four‑part architecture (HTTP server, Store, Raft, WAL), and explains how its built‑in leader election ensures that only one node acts as the master at any time.

It then walks through the creation of a three‑node ETCD cluster on CentOS 7, showing how to install ETCD via a build script, configure the service with a systemd unit file, and start the cluster.

Example configuration file ( etcd.conf) content:

ETCD_NAME=instance01
ETCD_DATA_DIR="/usr/local/etcd/data"
ETCD_LISTEN_CLIENT_URLS="http://10.143.74.108:2379,http://127.0.0.1:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://10.143.74.108:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.143.74.108:2380"
ETCD_LISTEN_PEER_URLS="http://10.143.74.108:2380"
ETCD_INITIAL_CLUSTER="instance01=http://10.143.74.108:2380,instance02=http://10.202.253.147:2380,instance03=http://10.202.254.213:2380"
ETCD_INITIAL_CLUSTER_STATE=new

The corresponding systemd unit file ( etcd.service) is also provided:

[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd
KillMode=process
Restart=always
RestartSec=3
LimitNOFILE=655350
LimitNPROC=655350
PrivateTmp=false
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

For high‑availability at the application level, the article presents a Go implementation that uses the clientv3 and concurrency packages to perform leader election.

Key Go snippets:

const prefix = "/nanoPing"
const prop = "local"
var leaderFlag bool

func campaign(c *clientv3.Client, election string, prop string) {
    for {
        s, err := concurrency.NewSession(c, concurrency.WithTTL(15))
        if err != nil { log.Println(err); continue }
        e := concurrency.NewElection(s, election)
        ctx := context.TODO()
        if err = e.Campaign(ctx, prop); err != nil { log.Println(err); continue }
        log.Println("elect: success")
        leaderFlag = true
        select {
        case <-s.Done():
            leaderFlag = false
            log.Println("elect: expired")
        }
    }
}

func run() {
    log.Println("[info] Service master")
    log.Println("[info] Task start.")
}

func Start() {
    donec := make(chan struct{})
    cli, err := clientv3.New(clientv3.Config{Endpoints: g.Config().Etcd.Addr, Username: g.Config().Etcd.User, Password: g.Config().Etcd.Password})
    if err != nil { log.Fatal(err) }
    defer cli.Close()
    go campaign(cli, prefix, prop)
    go func() {
        ticker := time.NewTicker(10 * time.Second)
        for {
            select {
            case <-ticker.C:
                if leaderFlag {
                    run()
                    return
                } else {
                    log.Println("[info] Service is not master")
                }
            }
        }
    }()
    <-donec
}

Testing steps show how the elected master node prints success messages, how a non‑master node logs its status, and how, after the master process exits, another node automatically becomes the new leader.

In summary, by leveraging ETCD's leader election and systemd's process supervision, the tutorial demonstrates a practical way to achieve service high availability, while also highlighting ETCD's broader capabilities such as shared configuration, service discovery, and the underlying Raft consensus algorithm.