Design and Implementation of Kuaishou's Metric Middle Platform

Kuaishou's Metric Middle Platform is a core component of the company's data platform, created to address data quality and development efficiency issues caused by fragmented metric management across BI, AB testing, and operations systems.

The platform adopts a Headless BI approach, providing unified metric definition, management, and service layers. It separates data collection, processing, and analysis, enabling consistent metric standards and reducing redundant development.

Metric management is organized into three layers: concept (metadata of metrics, dimensions, and tables), logic (data modeling to map raw metadata to efficient query structures), and application (datasets that group relevant metrics and dimensions for specific use cases).

Data modeling follows a three‑step process—conceptual modeling, logical modeling (including model discovery, field association, optimal path calculation, and indexing), and physical modeling (star or snowflake schemas). This ensures high‑performance query execution.

The unified analysis language OAX defines queries using five elements (data scope, metrics, dimensions, time range, filters), simplifying complex calculations such as dynamic granularity and table‑level operations.

OCTO, the unified query engine, translates OAX into federated queries across heterogeneous storage systems (e.g., ClickHouse, Hive, MySQL), performs query planning, optimization (rule‑based and cost‑based), and execution, supporting advanced analytics like window functions and comparative analyses.

An acceleration layer further improves performance by materializing frequently accessed metric‑dimension combinations into hot storage, either through manual configuration or automated analysis of query patterns, achieving up to ten‑fold speedups.

Since launch, the platform has covered all core business metrics (tens of thousands of metrics), served millions of daily queries, and demonstrated significant gains in data quality, efficiency, and cost reduction. Future work focuses on intelligent data retrieval using large language models and continued performance enhancements through vectorization and native SQL execution.