How ZooKeeper Supports HBase: Coordination, Fault Tolerance, Log Splitting, META Table Management, and Replication

ZooKeeper, as a distributed coordination component, plays a crucial supporting role in many big‑data systems, including HBase. This article uses the typical HBase setup to illustrate the functions ZooKeeper provides for HBase.

Below is a diagram of a complete HBase cluster architecture, showing the ZK nodes.

HMaster and RegionServer Fault Tolerance

When an HBase cluster starts, it registers the following znodes in ZooKeeper:

/hbase/master – contains information about the currently active (elected) HMaster.

/hbase/backup-masters/[host-name] – each child znode holds information about a hot‑standby HMaster.

/hbase/rs/[host-name] – each child znode holds information about a RegionServer.

All these znodes are ephemeral; HMaster and RegionServers maintain them via heartbeats. The active HMaster watches the /hbase/rs path; when a RegionServer fails, its heartbeat stops, the corresponding znode is removed, and the HMaster detects the failure and re‑routes the Regions to other servers. Similarly, backup HMasters watch /hbase/master; if the current HMaster crashes, the znode disappears, triggering a new election.

Log Split Management

When a RegionServer crashes, HBase must recover data that resides in the WAL (HLog) but has not yet been persisted to HFiles. To accelerate failover, HBase splits the HLog by Region into multiple fragments and assigns them to live RegionServers for replay.

The split‑log information is coordinated through a temporary node /hbase/splitlog in ZooKeeper, which stores the mapping between alive RegionServers and the Regions whose HLogs they should replay. Each RegionServer obtains its assignment from this node, replays the logs, and writes the result back, notifying the HMaster for further processing.

.META. Table Location Maintenance

HBase has a special table called .META. (the older -ROOT- table has been deprecated). This table, organized like a B‑tree, records the locations of all Regions in the cluster and never splits. ZooKeeper keeps a persistent node /hbase/meta-region-server that records which RegionServer hosts the .META. table.

When a client first connects to an HBase cluster, it queries this ZNode, then contacts the RegionServer holding .META. to obtain the Region location for the desired RowKey, caching the result for subsequent reads and writes.

If a Region moves, the client cache expires, or the server hosting .META. fails, the client can always retrieve the correct Region location via the ZooKeeper path, ensuring consistency.

Replication Management

HBase Replication synchronizes data from a primary cluster to one or more secondary clusters for disaster recovery and backup. After enabling replication, the primary cluster pushes data to the secondaries (asynchronously, synchronously, or serially) while guaranteeing eventual consistency. All replication state is stored in ZooKeeper under /hbase/replication, with three key znodes:

/hbase/replication/state – a boolean indicating whether replication is enabled.

/hbase/replication/peers – information about each secondary cluster and its status, each assigned a unique ID.

/hbase/replication/rs – indexes of all active RegionServers and HLog files in the primary cluster, plus read offsets.

The primary HMaster updates these znodes each time it pushes new data to a secondary, coordinating replication progress.

Multiple HBase clusters can share a single ZooKeeper ensemble by configuring different root paths (e.g., /hbase-cluster1, /hbase-cluster2) via the zookeeper.znode.parent parameter.

Conclusion

ZooKeeper provides essential coordination services for HBase, handling master/RegionServer failover, log splitting, META table location, and replication state, thereby ensuring high availability and consistency in large‑scale big‑data deployments.