Service Governance and etcd: Concepts, Raft & BoltDB Implementation, and Large‑Scale Practices at Baidu

Service governance, a critical part of modern cloud‑native and micro‑service architectures, covers registration and discovery, traffic monitoring, traffic scheduling, service control, and security; Baidu’s experience highlights the need for unified platforms to manage complex, large‑scale service ecosystems.

The core challenges of massive service systems are high reliability (99.99%+ availability), high performance (low latency under massive traffic), and high scalability (supporting millions of instances), all of which demand robust governance solutions.

etcd, originally developed by CoreOS in 2013, is a distributed KV store that uses the Raft consensus algorithm and BoltDB for persistent storage; it offers strong consistency, high availability, and a simple gRPC‑based client API, positioning it as a preferred alternative to ZooKeeper and Consul.

Raft in etcd defines four node states (Leader, Follower, Candidate, Pre‑Candidate). Leaders send periodic heartbeats, manage term numbers, and coordinate log replication; candidates undergo pre‑vote and election phases to become leaders, ensuring only one leader per term and guaranteeing safety through strict voting and log‑matching rules.

Log replication proceeds by the leader creating log entries for client proposals, broadcasting AppendEntries RPCs to followers, collecting acknowledgments (MatchIndex), computing the commit index once a majority have persisted the entry, and finally responding to the client, thereby maintaining strong consistency across the cluster.

BoltDB provides an efficient B+‑tree on a single memory‑mapped file, with meta, freelist, bucket, branch, and leaf pages; queries load meta data, locate buckets, traverse branch nodes to leaf nodes, and retrieve values. Example Go code demonstrates opening a read‑only transaction, accessing a bucket, and getting a key, while the search function shows recursive page traversal for key lookup.

Based on etcd, Baidu built the "Tianlu" service‑governance platform, comprising a registration center, visual management UI, SDKs, a unified gateway, and a mesher; it achieves high availability via single‑zone deployment with master‑slave failover and cache‑backed degradation, high performance through multi‑level caching and direct service calls, and high scalability to support hundreds of thousands of instances.

Operational metrics target >99.99% availability and <100 ms latency, with early fault detection via health checks, automated recovery using callbacks, and manual on‑call processes; the platform also offers trace integration, multi‑language SDKs, and real‑time policy updates.

In conclusion, while service governance is increasingly vital for cloud‑native enterprises, successful adoption requires mature platforms, deep technical expertise, and disciplined service design; the article provides practical insights and a reference implementation to help teams navigate these challenges.