How ZooKeeper Powers Distributed Systems: Architecture and Operations Explained

Big data processing frameworks Hadoop, Redis distributed service Codis, and Taobao's distributed message middleware MetaMQ all use ZooKeeper as a foundational component, demonstrating its power.

What is ZooKeeper

ZooKeeper (ZK) is an open‑source distributed coordination service framework, an implementation of Google’s Chubby, and a sub‑project of Hadoop.

It mainly solves consistency problems in distributed systems, encapsulating complex, error‑prone services and offering simple interfaces for high‑performance, stable external access.

Typical use cases include unified naming service, distributed configuration management, cluster management, distributed locks, distributed queues, and more.

Overall Structure

Externally, ZK appears as a single service accessed via its API.

Internally, ZK consists of a server cluster where each server holds identical data; one server acts as the leader to synchronize others.

When a client connects, ZK assigns the connection to an appropriate server based on load, making the assignment transparent to the client.

Each server maintains a hierarchical file‑system‑like namespace stored in memory for fast access, with data periodically persisted to disk for safety.

Workflow

Read

Read operations are simple because all servers hold identical data; a client’s read request is served directly from the memory of the server it is connected to, yielding very low latency.

Write

Write operations affect the namespace and node data, requiring coordination among servers to maintain consistency, thus more complex than reads.

Steps:

Client sends write request to its connected server.

Server forwards the request to the leader.

Leader performs the write, then notifies other servers to update.

After servers update, the leader informs the client that the write is complete.