How Flink’s Stream‑Batch Integration Powered Alibaba’s Record‑Breaking Double‑11

Alibaba’s 2020 Double‑11 event reached a record peak of 4 billion records per second and 7 TB of data per second, driven by Flink‑based real‑time computing.

Flink supported not only the GMV dashboard but also real‑time recommendation, ad anti‑fraud, order tracking, attack detection, and extensive monitoring, running over 35,000 jobs on more than 1.5 million CPU cores, placing Alibaba among the world’s leading real‑time platforms.

The new stream‑batch unified data applications debuted in Alibaba’s core data scenarios, proving stability, performance and efficiency under production pressure.

In typical marketing analytics, separate batch and streaming pipelines cause duplicated development and inconsistent metrics; a unified stream‑batch engine eliminates this gap, delivering identical real‑time and historical reports with a single code base.

Flink’s architecture treats batch as a bounded stream (“batch on streaming”), enabling one engine to handle both modes. Since version 1.9, Flink SQL provides native stream‑batch semantics, and from 1.11 the DataStream API adds batch capabilities and connectors for Kafka, HDFS, and other sources.

The unified engine also improves resource utilization: idle real‑time cluster capacity at night is reused for batch jobs, raising overall CPU usage without additional resources.

Historically, Hadoop/MapReduce handled batch and Storm handled streaming; later Spark added streaming but remained batch‑centric. Flink, originating from the Stratosphere project, chose a streaming‑first design, achieving tighter integration of batch and stream processing.

Alibaba adopted Flink in 2015 for search‑recommendation, scaling it through successive Double‑11 events, and later contributed to the open‑source community, acquiring Ververica and promoting Flink globally.

Flink Forward Asia 2020 showcased these advances, with talks from Alibaba, ByteDance, Tencent, Meituan, Kuaishou, Weibo and others, highlighting the growing ecosystem and real‑world impact of stream‑batch technology.