How ZooKeeper Implements Distributed Locks: Mechanism and Pitfalls

Apache ZooKeeper is an open‑source coordination service used for configuration, naming, synchronization and distributed locks. The article first describes the “herd effect” that occurs when many clients simultaneously watch a lock node: when the lock is released, all 2000 clients are notified and immediately try to create a node, causing a sudden surge of traffic that can overload the ZooKeeper server.

To avoid this, ZooKeeper’s temporary sequential nodes are used. Each client creates an EPHEMERAL‑SEQUENTIAL node under a lock path; the nodes are ordered by their sequence numbers. Clients watch only the node that precedes them (chain watching). When the smallest node releases the lock and deletes its node, only the next node receives the notification, eliminating the herd effect.

The article illustrates the process with three example threads (lock‑1, lock‑2, lock‑3). lock‑2 watches lock‑1, lock‑3 watches lock‑2. When lock‑1 finishes and deletes its node, ZooKeeper notifies lock‑2, which then acquires the lock because its sequence number is now the smallest.

A failure scenario is also presented: if a middle node (e.g., lock‑2) crashes, ZooKeeper removes its node, leaving lock‑3 without a predecessor to watch. The client must detect the missing predecessor, find the next alive node with a smaller sequence number (lock‑1), and re‑establish the watch. This “break‑and‑reconnect” logic is essential for lock stability.

Because implementing chain watching and reconnection is complex, most production systems use the Curator library, which encapsulates the fault‑tolerant logic for ZooKeeper distributed locks.

Summary points: (1) ZooKeeper implements distributed locks with temporary sequential nodes; (2) chain watching plus break‑and‑reconnect logic ensures low notification traffic and fault tolerance; (3) Curator is the common practical choice.