HBase RowKey and Index Design: Principles, Practices, and Case Studies

The presentation begins with an overview of HBase architecture, covering tables, regions, column families, RegionServers, MemStore, and HFile, and explains how RowKey influences data distribution and read/write routing.

It then emphasizes the importance of systematic demand research, identifying load characteristics, query scenarios, and data properties to inform RowKey and index design.

Core RowKey and secondary index design principles are discussed, including uniqueness, alignment with frequent query patterns, and strategies to avoid data hotspots.

Three key techniques for mitigating hotspot issues are detailed: (1) Reversing the RowKey, (2) Salting with random bytes, and (3) Hashing portions of the RowKey, each with advantages and trade‑offs for scan performance.

The document explains HBase’s data partitioning methods (hash vs. range) and outlines two secondary index models—global and local—highlighting their performance implications.

Design guidelines for selecting leading columns, ordering composite keys, and adding auxiliary columns are provided to optimize query selectivity and storage efficiency.

Three real‑world case studies illustrate the concepts: OpenTSDB’s time‑series model with salted RowKey, JanusGraph’s vertex‑centric RowKey layout, and GeoMesa’s spatio‑temporal indexing using Z‑order and XZ‑order schemes.

The summary reiterates the four main takeaways: HBase basics and RowKey role, demand‑driven design dimensions, RowKey/index design techniques, and practical case‑based RowKey structures for diverse workloads.