How Hologres + FBI Powers Real‑Time Experience Insight at Alibaba: A Deep Dive

CCO (Chief Customer Officer) at Alibaba focuses on improving user experience by combining voice data with transaction, logistics, and refund data. In March, over 80 business analysts submitted more than 22,000 queries, consuming an estimated 458 workdays, highlighting the need for a flexible ad‑hoc query capability.

Solution Overview The team iterated on data products, ultimately adopting Hologres + FBI to support over 50 billion daily detail‑level aggregations with sub‑second response times, boosting analyst productivity by more than tenfold.

Stage 1: Pre‑computed MOLAP Cubes + ADB Acceleration

This approach used MaxCompute to pre‑compute multi‑dimensional aggregates stored in an ADS layer and queried via OLAP engines such as ADB or MySQL. While suitable for stable dimension sets, it suffered from poor flexibility, data explosion, high industry‑re‑refresh cost, inaccurate UV deduplication, and long data back‑flow times.

Stage 2: Lightly Summarized Fact Tables + Dimension Joins

To enable real‑time insights, the design abandoned pre‑aggregation and stored lightweight summary fact tables keyed by entity IDs (product, seller, buyer, rider). These tables were joined with dimension tables in the OLAP engine for multi‑dimensional analysis. This improved flexibility and extensibility but required high‑performance OLAP engines and still struggled with large‑scale joins, UV deduplication, and certain snapshot features.

Stage 3: Detail‑Wide Tables for Ad‑hoc Queries

The final architecture writes raw detail data directly into Hologres, allowing ad‑hoc queries that join dimension tables at query time. This supports rich real‑time and offline federated queries, flexible window comparisons, and precise UV deduplication using Hologres' columnar storage, RoaringBitmap, and APPROX_COUNT_DISTINCT.

Key implementation details include:

Table design: columnar storage with business PK and daily partitioning.

Table Group and index choices (distribution, clustering, bitmap) to optimize join performance.

Dynamic partition handling (T+1 partition overwrite) using Flink sink configuration.

FBI Velocity syntax for query pruning based on selected dimensions.

RoaringBitmap extensions for efficient distinct counting on high‑frequency dimensions.

Approximate distinct counting (APPROX_COUNT_DISTINCT) for less critical dimensions.

Operational results show that the integrated insight platform processes over 50 billion rows per month with second‑level query latency, supporting 100+ business analysts during major promotions and saving an estimated 6,875 person‑days during a single Double‑11 event.

Future Directions

Unify streaming and batch pipelines into a true Lambda‑free architecture using Hologres as the single storage and compute layer.

Migrate all FBI datasets to a managed data‑service platform for centralized monitoring, performance alerts, and resource optimization.