Interview with Wang Feng (Mo Wen): The Future of Apache Flink and Streaming Warehouses

Author: Cai Fangfang

Interviewee: Wang Feng (alias Mo Wen), Alibaba open‑source big‑data platform lead.

InfoQ's 2021 technology trend outlook highlighted the acceleration of integration in big data, and Apache Flink is taking a new step toward a “Streaming Warehouse”.

At Flink Forward Asia 2021, Wang Feng presented the concept of a Streaming Warehouse, arguing that Flink should evolve from pure stream processing to cover broader data‑warehouse scenarios.

He explained the “stream‑batch integration” (流批一体) philosophy, its recent technical advances in Flink 1.14 (mixed bounded/unbounded streams, unified Source/Sink API, combined DataStream and Table APIs), and two representative use cases: full‑incremental data integration with Flink CDC and a unified real‑time‑offline data‑warehouse architecture.

Flink CDC now supports many databases (MySQL, PostgreSQL, MongoDB, Oracle, etc.) and enables one‑SQL‑statement incremental sync.

The emerging “Streaming Warehouse” aims to provide end‑to‑end real‑time analytics with a single API, eliminating the split between streaming and batch pipelines and reducing system complexity.

To realize this, the community is developing a “Dynamic Table” storage layer that offers both a file store for batch reads and a log store for streaming updates, fully integrated with Flink SQL.

Wang forecasts that a mature Streaming Warehouse solution will appear within a year, with a preview expected in Flink 1.15, and emphasizes that Flink’s strength lies in its stateful streaming combined with a unified storage system.

Overall, the interview underscores the industry trend toward integrated, one‑stop data‑processing platforms and positions Flink’s upcoming features as a key driver of that evolution.