Comprehensive Overview of ZooKeeper: Architecture, Features, and Use Cases

1. What is ZooKeeper?

ZooKeeper is an open‑source distributed coordination service, an implementation of Google’s Chubby, acting as a cluster manager that monitors node states and provides simple, high‑performance APIs to clients.

2. What does ZooKeeper provide?

It offers a hierarchical namespace (similar to a file system) and a notification mechanism for clients.

3. ZooKeeper file system

ZooKeeper maintains a multi‑level node namespace (znodes) where each node can store data; unlike traditional file systems, directories can also hold data. The tree is kept in memory, limiting each node’s data size to about 1 MB.

4. Four types of znodes

PERSISTENT : Remains after the client disconnects.

PERSISTENT_SEQUENTIAL : Persistent with an automatically assigned sequential suffix.

EPHEMERAL : Deleted automatically when the client disconnects.

EPHEMERAL_SEQUENTIAL : Ephemeral with a sequential suffix.

5. ZooKeeper watch mechanism

Clients can set a watcher on a znode; when the znode changes, ZooKeeper sends a one‑time notification to the client.

6. What does ZooKeeper do?

It provides naming services, configuration management, cluster management, distributed locks, and queue management.

7. Naming service (file system)

ZooKeeper creates a global path that can be used as a unique name to locate resources or services in the cluster.

8. Configuration management

Configuration data is stored in znodes; changes trigger watchers, allowing clients to update their configuration dynamically.

9. Cluster management

Machines create temporary znodes under a parent directory; the creation or deletion of these nodes signals machine joins or failures, enabling leader election based on the smallest sequential node.

10. Distributed lock

Locks are implemented by creating a designated lock znode; the client that successfully creates it holds the lock, and releasing the lock is done by deleting the node. A sequential lock variant uses the smallest sequential node to grant the lock.

11. Distributed lock acquisition process

Clients create a temporary sequential node under a lock directory, list all children, and if their node has the smallest sequence number they acquire the lock; otherwise they watch the next smaller node and repeat until they become the smallest.

12. Queue management

Two queue types are supported: a synchronized queue that becomes usable only when all members are present, and a FIFO queue where clients create sequential znodes and consume the smallest node.

13. Data replication

ZooKeeper replicates data across all servers for fault tolerance, scalability, and performance. It uses a "write any" model where any server can accept writes, with replication modes ranging from write‑master to write‑any.

14. Working principle

The core is an atomic broadcast (Zab protocol) that ensures consistency among servers. Zab operates in recovery mode (leader election) and broadcast mode (synchronization).

15. Transaction ordering

Each proposal receives a 64‑bit zxid (Zookeeper Transaction Id) composed of an epoch and a counter, guaranteeing total order of updates.

16. Server states

Servers can be LOOKING (searching for a leader), LEADING (the elected leader), or FOLLOWING (synchronizing with the leader).

17. Leader election

ZooKeeper uses either basic Paxos or fast Paxos; the default is fast Paxos, where servers propose themselves as leader and the one with the highest zxid wins.

18. Synchronization process

After leader election, followers send their highest zxid to the leader, which determines a sync point, updates followers, and marks them as up‑to‑date.

19. Distributed notification and coordination

Clients modify znodes to trigger notifications; other clients watching those znodes receive updates, enabling real‑time progress monitoring.

20. Why is there a leader?

A leader is needed to execute exclusive business logic, reducing duplicate work and improving performance.

21. Handling node failures

With a quorum of at least three servers, ZooKeeper continues operating when one node fails; if the leader fails, a new leader is elected.

22. ZooKeeper vs. Nginx load balancing

ZooKeeper’s load balancing is programmable, while Nginx provides static weight‑based balancing; Nginx generally offers higher throughput.

23. Watch mechanism details

Watches are one‑time triggers sent asynchronously; they can be set on data or children, and are re‑registered upon client reconnection. Certain edge cases may cause a watch to be lost.