Why ZooKeeper? Unveiling the Core of Distributed Coordination Services

What Is ZooKeeper?

ZooKeeper is a popular distributed coordination service used to simplify the development of distributed applications by shielding developers from low‑level details and exposing a simple API.

Why Was ZooKeeper Created?

When multiple processes need to cooperate across different machines, the business logic becomes tangled with complex coordination code. ZooKeeper offers a common infrastructure so developers can focus on business logic rather than coordination details.

Key Characteristics of ZooKeeper

Simple API inspired by file‑system operations.

Runs on dedicated servers, separating it from business logic, which provides high fault‑tolerance and scalability.

What ZooKeeper Stores

It stores 协作数据 (metadata) rather than application data, which should be kept in dedicated storage such as HDFS.

Conceptually, ZooKeeper can be seen as a 特殊的 FS (special file system).

Application data and metadata have different consistency and durability requirements and should be treated and stored separately.

ZooKeeper’s Mission

ZooKeeper aims to solve the core problems of distributed application development by providing an efficient, reliable coordination service that offers:

Unified naming service.

Distributed locks.

Process crash detection.

Leader election.

Configuration management with timely propagation to clients.

Types of Multi‑Process Collaboration

Cooperation : Multiple processes must work together on a task (e.g., master‑slave task distribution).

Competition : Only one process may perform an action at a time (e.g., leader election after a master failure).

Cross‑Network Multi‑Process Collaboration

When processes are distributed across different hosts, ZooKeeper addresses three common challenges:

Message latency (out‑of‑order delivery).

Processor performance (delayed handling after receipt).

Clock skew between hosts.

ZooKeeper is carefully designed to hide these three issues, making them transparent to the application layer.

ZooKeeper Features

Sequential consistency: client requests are executed in the order they are issued.

Atomicity: a transaction is applied to all nodes or none.

Single view (eventual consistency) for all clients.

Reliability: successful transactions are permanently recorded.

Real‑time guarantees (eventual consistency) for read‑after‑write.

Design Goals

High performance: tree‑structured data nodes, all nodes kept in memory, followers and observers handle read‑only requests.

High availability: the service remains operational as long as a majority of machines are alive, with automatic leader election.

Sequential consistency: every transaction is forwarded to the leader and assigned a globally unique, monotonically increasing ID (zxid).

Final consistency: proposal voting ensures that a majority of nodes see the latest data after a transaction commits.

What Came Before ZooKeeper?

Prior to ZooKeeper, distributed systems typically used either a distributed lock manager or a distributed database to achieve process coordination.

When Not to Use ZooKeeper

ZooKeeper is not suitable for massive data storage; it is designed to store metadata, not large application data, which should be kept in dedicated storage solutions.