Introduction to Apache ZooKeeper: Concepts, Setup, and Usage Guide

Apache ZooKeeper is a high‑performance coordination server for distributed applications, offering simple interfaces for naming, configuration management, synchronization, locking, and leader election, thus relieving developers from implementing these services from scratch.

Key coordination services provided by ZooKeeper include:

Name Service – maps names to information, similar to DNS, and can be extended to group membership information.

Locking – provides a simple way to implement distributed mutexes.

Synchronization – supports producer‑consumer queues, barriers, and other synchronization primitives.

Configuration Management – stores centralized configuration data that new nodes can read immediately upon joining.

Leader Election – enables automatic fail‑over by electing a leader among nodes.

ZooKeeper itself follows a client‑server model where a set of servers forms an ensemble. Clients connect to one server at a time; the server replies to pings to indicate liveliness, and sessions transparently fail over to another server if needed. The ensemble uses a quorum (strict majority) to guarantee write consistency.

Data in ZooKeeper is organized as a hierarchical namespace of znodes , similar to a file system. Each znode can hold up to 1 MB of data and is stored in memory on each server for fast reads, while writes are logged to disk and require a majority of servers to acknowledge.

When scaling the ensemble, read performance remains stable, but write latency increases with more nodes because the write must be replicated to a majority.

Setting up a three‑node ZooKeeper ensemble (ZooKeeper 3.4.5)

1. Install JDK on each node.

2. Download and extract ZooKeeper:

wget http://www.bizdirusa.com/mirrors/apache/ZooKeeper/stable/zookeeper3.4.5.tar.gz
 tar xzvf zookeeper3.4.5.tar.gz

3. Create a data directory:

mkdir /var/lib/zookeeper

4. Create a configuration file conf/zoo.cfg similar to:

tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zkserver1.mybiz.com:2888:3888
server.2=zkserver2.mybiz.com:2888:3888
server.3=zkserver3.mybiz.com:2888:3888

5. Create a myid file on each server containing its numeric ID (1, 2, or 3).

6. Start each server:

zookeeper3.4.5/bin/zkServer.sh start

7. Use the CLI to interact with the ensemble:

zookeeper3.4.5/bin/zkCli.sh -server zkserver1.mybiz.com:2181,zkserver2.mybiz.com:2181,zkserver3.mybiz.com:2181

Example CLI commands:

[zk:127.0.0.1:2181(CONNECTED) 2] create /mynode helloworld
Created /mynode
[zk:127.0.0.1:2181(CONNECTED) 6] get /mynode
helloworld ...
[zk:127.0.0.1:2181(CONNECTED) 7] rmr /mynode
[zk:127.0.0.1:2181(CONNECTED) 10] create /mysecondnode hello
Created /mysecondnode
[zk:127.0.0.1:2181(CONNECTED) 12] get /mysecondnode 1
hello ...
[zk: localhost:2181(CONNECTED) 1] set /mysecondnode hello2
... (watch notification)
[zk:127.0.0.1:2181(CONNECTED) 13] WATCHER::
WatchedEvent state:SyncConnected type:NodeDataChanged path:/mysecondnode

Clients can also create child znodes, retrieve statistics with stat /mysecondnode , and use language bindings (Java, C, Python, etc.) to integrate ZooKeeper into applications.

ZooKeeper is widely used by projects such as Apache Hadoop, HBase, Accumulo, Solr, Mesos, Neo4j, and Cloudera Search for high‑availability and coordination tasks.

In conclusion, ZooKeeper provides a stable, simple, and high‑performance coordination service that saves developers from reinventing distributed protocols, making it easier to build reliable distributed systems.