Kuaishou Big Data Security Platform: Architecture, Governance, and Practices

Kuaishou, founded in 2011, serves billions of daily and monthly active users, requiring a robust data platform that handles petabyte‑scale data (EB level). The platform aims to improve decision‑making efficiency through a data middle‑platform that provides data warehouses, services, analysis, experiments, and AB testing.

The presentation focuses on data security, outlining five major sections: background introduction, platform construction, governance practice, outcomes and planning, and a Q&A.

Background: Kuaishou's data security platform protects the full data lifecycle, from warehouse construction (ODS, data marts, dimension tables) to data ingestion (encryption, masking) and data services (user authentication).

Challenges: Supporting over 30 systems, fine‑grained control across resources, accounts, and operations, high availability for authentication services, and extensibility for diverse business needs.

Construction Approach: Established data and information security committees, defined classification and permission standards, and built a dedicated security platform team. Principles balance security and efficiency with tiered approval processes.

System Architecture: Multi‑layer design with an application layer, a core security platform layer (plugins, interfaces, services, storage), and a dependency layer (tenant/account systems). Core modules include plugin layer (engine‑specific auth), interface layer (HTTP/RPC), service layer (resource and account integration), and storage layer (caching and acceleration).

Key Technologies: Authentication system (lightweight, localized, evolvable) using three‑step token exchange and supporting multiple token types; permission models combining RBAC, PBAC, and a custom PRBAC strategy (subject, resource, action, condition). Unified authorization for both application and big‑data engine workloads.

Full‑Link Audit: End‑to‑end logging across production, application, Hive, HDFS, with lineage integration, unified format, and real‑time risk alerts.

Data Classification & Grading: Classified into public (C1‑C4) and privacy (P1‑P4) levels with corresponding protection measures, automated metadata collection, algorithmic detection, and rule‑based tagging, followed by manual confirmation.

Data Engine Security: Addressed lack of account systems, missing audit capabilities, and operational governance gaps through refined account structures, fine‑grained row‑level permissions, and engine‑specific auth plugins.

Sensitive Data Protection: Introduced secure isolation warehouses, encryption, field‑level access control, and automated scanning to identify and protect sensitive information.

Results & Planning: Deployed across 30+ systems, handling millions of resources and thousands of daily requests, with stable operation. Future plans include 100% engine authentication coverage, enhanced situational awareness, exploration of advanced privacy technologies, and AI‑driven data classification.

Q&A: Covered tokenized data ingestion, cross‑department permission workflows, and row‑level deletion using Hudi.