Kuaishou Analytics Service 3.0: Architecture, Evolution, and Practice

Introduction – The talk introduces Kuaishou's analytics domain, outlining the business background of data analysis, a typical use case (province‑level GDP statistics), and the three‑stage data processing pipeline: data ingestion, dataset modeling, and metric calculation.

Analytics Service Evolution – Four development phases are described: (1) 2019 tool‑based stage, (2) 2020‑2021 Service 1.0 platformization, (3) 2021‑2022 Service 2.0 standardization with the Gaia metric middle‑platform, and (4) the current Service 3.0 unification of standard and non‑standard services.

Service 1.0 – Platformization – Challenges included low user efficiency (SQL knowledge required) and high R&D cost of maintaining eight separate tools. The solution unified product and service layers, introduced a one‑stop analysis platform, and abstracted physical data into reusable datasets.

Service 2.0 – Standardization – To address data quality and processing inefficiency of user‑defined datasets, a standard metric middle‑platform was introduced, providing unified definition, processing, and consumption of key indicators.

Service 3.0 – Unified Architecture – The architecture consists of three layers: data layer, analysis service layer (including a unified analysis engine and a unified query engine), and application layer. The unified analysis engine offers a three‑tier design (data preparation, acceleration, service) and exposes the OAX (Open Analysis Expression) language. The unified query engine provides a three‑tier design (adapter, query, access) and introduces the FQL (Federation Query Language) for cross‑source queries.

Data Acceleration Techniques – Three mechanisms are employed: intelligent cache with proactive pre‑heating, model materialization (analysis, acceleration, and application phases), and query optimization (predicate push‑down, column pruning, bitmap group‑by, local query execution). These are built on Apache Calcite and Arrow.

Benefits of the Unified Engines – Data decoupling, improved consumption and production efficiency, and higher data quality through unified metric definitions. The unified analysis engine also supports advanced calculations such as LOD, INCLUDE, FIXED, and EXCLUDE.

Performance and Adoption – Service 3.0 achieves four‑nine availability, >20% query performance improvement (P90 ≈ 15 s), and serves over 50 applications with more than 5 million daily queries.

Future Outlook – The roadmap focuses on intelligent analysis (predictive and role‑based analytics, large‑model integration), intelligent diagnosis (attribution and performance suggestions), and intelligent acceleration (materialization for hot‑path queries).

Speaker – The session is presented by Qian Jia, Kuaishou Big Data Platform Technical Expert, with editorial support from Baotingwen and proofreading by Li Yao.