Real-time Multi-dimensional Analytics at ZhongAn: Practices, Challenges, and Technology Choices
This article presents ZhongAn Insurance's experience building real-time multi-dimensional analytics, covering application scenarios, technical challenges, the evolution of their architecture from offline to Flink‑ClickHouse and finally to StarRocks, and the principles guiding their technology selection.
The article introduces ZhongAn's real-time multi-dimensional analytics practice, outlining why real-time analysis is needed for online business monitoring and marketing, and describing two concrete use cases: product‑placement dashboards and user‑behavior analysis.
It then discusses the main difficulties encountered, such as supporting data updates in OLAP systems, balancing update latency with query performance, and integrating real‑time and batch data streams.
The evolution of the solution is detailed: starting with offline reporting on MaxCompute, moving to ClickHouse for interactive analysis in 2018, adopting Flink+MySQL for early real‑time workloads in 2019, then Flink+ClickHouse, and finally introducing StarRocks in 2022 to address update performance and simplify operations.
Key technical decisions are explained, including the use of ClickHouse's ReplacingMergeTree engine for versioned updates, partitioning and hashing strategies to ensure data locality, and mechanisms for handling schema unification via views that union real‑time and batch tables.
The article evaluates StarRocks against ClickHouse, showing that StarRocks offers faster real‑time updates and more stable query performance, while ClickHouse retains richer analytical functions.
Finally, it shares lessons on technology selection—favoring simplicity, reliability, and foresight—and provides practical tips for deploying and operating the Flink‑StarRocks stack in a production BI platform.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
DataFunSummit
Official account of the DataFun community, dedicated to sharing big data and AI industry summit news and speaker talks, with regular downloadable resource packs.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.