How Alibaba’s CCO Built a Cloud‑Native Real‑Time Data Warehouse with Hologres

In the 2020 Double‑11 event, Alibaba’s Customer Experience (CCO) team combined MaxCompute interactive analysis with Flink to create a cloud‑native real‑time data warehouse using Hologres, marking a new record for the big‑data platform.

Real‑time Architecture Evolution

Database Stage

Initially a Lambda architecture where data flowed from collection to processing to service, stored in OLTP and KV engines, and served via point queries. Flink performed multi‑stream joins and HBase provided dimension table joins. This end‑to‑end approach was flexible and low‑cost but became unsustainable as data complexity grew.

Traditional Data Warehouse Stage

To address scaling issues, an OLAP engine (AnalyticDB) was introduced, and data was layered into DWD and DWS stages, stored in Lindorm and later synchronized via Hlog. While this improved flexibility and reuse, it introduced task duplication, storage redundancy, and high operational costs.

Real‑time Data Warehouse Stage

Hologres was selected for its unified row‑store and column‑store capabilities, high‑throughput real‑time writes, and seamless Binlog subscription. It eliminated task duplication, reduced storage redundancy, and simplified metadata management.

Real‑time Data Warehouse Architecture

Unified storage: Row‑store tables handle point queries, while column‑store tables serve OLAP queries.

Simplified real‑time pipeline: Hologres Binlog subscription replaces separate Lindorm subscriptions, reducing data flow complexity.

Unified service routing: Point queries are routed to row‑store tables, OLAP queries to column‑store tables.

Stream‑batch integration: External tables enable federated joins, allowing real‑time OLAP without data duplication.

Business Value

Real‑time architecture upgrade: Over 60% of real‑time jobs now run on the new warehouse, achieving peak‑shaving and cost reduction.

Self‑service analytics: FBI + Vshow + Hologres dashboards provide instant, ad‑hoc analysis for business users.

Flexible indexing and table‑group optimization: Distribution, clustering, and segment keys improve query performance; shard‑count tuning reduces SQL latency.

Service resource systematization: Thousands of monitoring screens enable efficient staffing and real‑time alerts.

Experience engine intelligence: Integrated multi‑channel data accelerates root‑cause analysis, cutting issue‑resolution time to minutes.

Cost savings: During Double‑11, the new architecture saved roughly 30% of infrastructure costs, amounting to millions of yuan.

Future Outlook

The team plans to deepen stream‑batch integration, explore HSAP capabilities, and continue co‑development with Hologres to further enhance scalability, reliability, and business impact.