Kuaishou Metric Middle Platform: Architecture, Standardization, and Core Technologies

The Kuaishou Data Platform shared its metric middle platform at the 2022 DataFun conference, summarizing best practices and technical implementations. The platform has undergone three years of iteration, built on a unified semantic layer concept.

Main contents include:

01 Introduction to Kuaishou Metric Middle Platform

02 Metric Standardization Management

03 Core Technologies of Unified Metric Service

04 Future Outlook

05 Q&A Session

1. Overall Introduction The metric middle platform serves as the semantic layer of Kuaishou's BI system, abstracting underlying data source models into metric‑oriented semantic models, enabling "define once, use everywhere" across dashboards, AB testing, decision systems, and vertical data systems.

2. Metric Standardization Management Before the platform, over 100,000 tables and duplicated metrics caused inconsistencies. In 2021, a standardization project was launched, establishing organization, tools, and specifications. The process includes defining business scope, calculation logic, and dataset planning, ensuring unified naming, consistent definitions, and streamlined management.

3. Core Technologies

3.1 Headless BI – Metrics are defined once and can be used in dashboards and automated tools, breaking the limitation of traditional BI where metrics are confined to BI tools.

The three goals of Headless BI are: low‑cost unified metric definition and management, large‑scale metric‑based query capability, and a unified API for integration with various downstream applications.

3.2 Automated Modeling – With 2,780 fact tables and 570 dimension tables, manual modeling is infeasible. The platform automatically builds star/snowflake models by leveraging dimension metadata, reducing effort, reusing relationships, and automatically updating models when source tables change.

3.3 Unified Metric Service – Provides metadata services, query services, and acceleration layers. Queries are expressed in OAX (Open Analysis eXpressions) and compiled through logical model planning, physical planning, and physical execution using the Octo federated query engine.

The architecture consists of an adaptation layer for heterogeneous data sources, a computation layer for federated calculations, and an interface layer offering the FQL (Federation Query Language) and Dataframe protocol based on Apache Arrow.

4. Future Outlook The platform will continue to improve intelligent modeling (auto‑inferring relationships) and intelligent acceleration (automatic pre‑aggregation based on query patterns).

5. Q&A The session addressed the division of responsibilities among business, data, and metric platforms, support for multi‑to‑many relationships, the role of standardized dimension IDs, metric types (atomic, derived, composite), and reasons for building a custom federated query engine instead of relying solely on ClickHouse or Presto.

Overall, the Kuaishou metric middle platform demonstrates a comprehensive solution for metric standardization, headless BI, automated modeling, and unified query services in a large‑scale data environment.