Big Data 23 min read

Inside Kuaishou’s Company‑Wide Metric Platform: Architecture, Lessons & Best Practices

This article details Kuaishou’s three‑year evolution of its metric middle platform, covering the data infrastructure, key challenges of data inconsistency and low analysis efficiency, the enterprise‑level OneMetric solution, architectural design, development phases, practical lessons, system implementation, and real‑world applications.

Kuaishou Big Data
Kuaishou Big Data
Kuaishou Big Data
Inside Kuaishou’s Company‑Wide Metric Platform: Architecture, Lessons & Best Practices

Introduction

Kuaishou presented its metric middle platform at the 2022 DataFun "Kuaishou Metrics Middle Platform" forum, sharing best‑practice technical solutions. This article is the first in a series that documents those insights.

01 Kuaishou and Kuaishou Big Data

Kuaishou is a digital community platform with a mission to enhance individual happiness. By 2022 Q3 it had 363 million daily active users and 626 million monthly active users. To support rapid business growth, Kuaishou built a company‑wide data platform whose purpose is to improve decision‑making efficiency and drive performance through data.

The platform’s data foundation consists of large‑scale storage and compute engines, a one‑stop data collection and development toolchain, and a unified data warehouse that aggregates all company data.

Metric Domain Overview

Metrics provide a semantic abstraction layer above physical data, enabling a "one definition, multiple uses" model. A metric is a business measurement, a dimension is the perspective (e.g., date or city), and an operator (e.g., YoY) performs secondary calculations.

Three reasons make a metric system essential:

It creates a semantic layer that hides physical‑engine differences.

It unifies definitions, preventing inconsistent terminology.

It serves as the data expression for analysis, appearing throughout the data chain.

Two Core Challenges in Big Data

Challenge 1 – Data Inconsistency (Quality): Siloed data warehouses and metric services lead to isolated data islands and divergent definitions.

Challenge 2 – Low Analysis Efficiency (Speed): Manual hand‑offs across requirement, scheduling, definition, development, verification, and delivery cause long delivery cycles.

Enterprise‑Level Metric Middle Platform (OneMetric)

To address these challenges, Kuaishou built OneMetric, which drives downstream data production and upstream analysis through unified metric metadata and services.

Key capabilities include:

Automatic generation of aggregation tables (No‑ETL) for downstream consumption.

Unified metric service (OneService) that standardizes metric consumption across applications.

System Architecture

The platform defines a four‑layer data model:

Data Source – physical tables, MySQL, ES indexes, etc.

Data Table – logical schema abstraction of sources.

Data Model – relationships between tables forming a semantic model.

Data Set – collections of models and metrics that expose headless BI capabilities.

These layers support both analysis‑side and production‑side pipelines.

Analysis Side

A unified query engine (Octo) handles cross‑engine distributed queries. On top of Octo, a semantic layer builds metric models, and a headless BI service (Data Set) provides unified access to data, metrics, and dimensions via the open analysis language OAX.

Metric‑BI integration creates Kuaishou’s distinctive BI experience.

Low‑code development enables rapid vertical application building.

Open APIs form a headless BI ecosystem.

Production Side

Unified production services translate metric‑centric requests into physical jobs dispatched by a workflow scheduler. Two production modes exist:

Automated production – identifies high‑value queries from BI/AB scenarios and automatically generates aggregation pipelines.

Manual production – designers create models and aggregation tables based on metric dimensions.

Development Phases and Lessons

Kuaishou’s metric platform evolved through four stages:

Exploration (2018) – early experiments without a clear direction.

Analysis‑Oriented Construction (2020) – built metric management and services to address growing data challenges.

Full Promotion – rolled out platform company‑wide, achieving universal adoption.

Production‑Oriented Construction – tackled AB testing and automated metric production.

Key lessons:

Metric metadata must drive the data chain, not just be stored.

For mature data warehouses, an analysis‑first approach is more suitable.

Practical Impact

Since deployment, Kuaishou has achieved:

High‑quality data services with no major quality incidents.

More than 10× efficiency improvement.

Significant cost reduction through data reuse.

Over 30 k core metrics covering all major business lines.

30+ downstream applications and 3 million daily queries.

300+ AB metrics in production, with ongoing automation.

QA Highlights

Common questions address alternative metric‑store products, handling of new business requirements, consistency between warehouses and metrics, and the suitability of a unified versus siloed metric platform. Answers emphasize that a unified platform ensures consistent definitions, reduces duplication, and complements—rather than replaces—existing data warehouses.

Conclusion

Kuaishou’s metric middle platform demonstrates how a company‑wide semantic layer can unify metric definitions, accelerate analysis, and automate production, providing a blueprint for other enterprises seeking similar data‑driven transformation.

data engineeringanalyticsBig DataKuaishoumetric platform
Kuaishou Big Data
Written by

Kuaishou Big Data

Technology sharing on Kuaishou Big Data, covering big‑data architectures (Hadoop, Spark, Flink, ClickHouse, etc.), data middle‑platform (development, management, services, analytics tools) and data warehouses. Also includes the latest tech updates, big‑data job listings, and information on meetups, talks, and conferences.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.