How ZooKeeper Powers Distributed Coordination: Core Concepts Explained

ZooKeeper Overview

ZooKeeper is an open‑source distributed coordination service that provides primitives for synchronization, configuration maintenance, and naming.

Design Goals

1. Final Consistency : Clients see a single consistent view regardless of the server they connect to.

2. Reliability : Once a message is accepted by one server, it is replicated to all servers.

3. Timeliness : Clients receive updates or failure notifications within a bounded time interval.

4. Wait‑free : Slow or failed clients do not block fast clients.

5. Atomicity : Updates either succeed completely or fail, with no intermediate state.

6. Ordering : Global order ensures that if message a precedes message b on one server, the order is preserved on all servers; partial order applies to messages from the same sender.

Data Model

ZooKeeper maintains a hierarchical namespace similar to a file system, where each node is called a znode and is identified by its full path.

Key characteristics of znodes:

Each znode can have child nodes and store data; EPHEMERAL nodes cannot have children.

Each znode is versioned; the version number increments with each data change.

Types: Persistent, Ephemeral, Non‑sequential, Sequential.

Watches can be set on znodes to receive notifications of data or child changes.

Every state change generates a globally ordered transaction ID (zxid).

Session Management

Clients establish a session with the ZooKeeper ensemble; if a client loses connection, it enters the CONNECTING state and attempts reconnection. Session expiration is determined by the server, not the client.

Watch Mechanism

A watch is a one‑time trigger sent to the client that set it when the watched data changes. Watches are set via getData(), getChildren(), or exists() and must be re‑registered after firing.

Consistency Guarantees

ZooKeeper provides sequential consistency, atomicity, a single system image, reliability, and timeliness for read and write operations.

Architecture and Roles

Servers assume one of three roles: leader, follower, or observer, and can be in states: leading, following, observing, or looking.

The core protocol is Zab (ZooKeeper Atomic Broadcast), which ensures ordered state updates via a two‑phase commit.

Leader Election

When the current leader fails, servers enter recovery mode and elect a new leader using either basic Paxos or fast Paxos. The algorithm selects the server with the highest zxid that obtains a quorum of votes.

Leader Workflow

Recover data.

Maintain heartbeats with followers and process their requests.

Handle follower messages: PING, REQUEST, ACK, REVALIDATE.

Follower Workflow

Send requests to the leader (PING, REQUEST, ACK, REVALIDATE).

Process messages from the leader.

Forward client write requests to the leader for voting.

Return results to the client.

Zab Protocol Details

When a server receives a request, the leader broadcasts a PROPOSAL to followers. Followers write to disk and ACK. Once a quorum of ACKs is received, the leader sends a COMMIT, guaranteeing ordered execution across the ensemble.

Summary

The article introduced ZooKeeper’s fundamentals, data model, session and watch mechanisms, consistency guarantees, leader election, server workflows, and the Zab protocol that underpins its reliable distributed coordination.