Why Alibaba Cloud Leads China’s Real‑Time Lakehouse Market in 2024

IDC recently released its first "IDC MarketScape: China Real‑Time Lakehouse Market 2024 Vendor Assessment" (Doc# CHC51768224, July 2024), naming Alibaba Cloud as a leader in the report’s inaugural publication.

Lakehouse architecture is gaining widespread adoption, offering an open data framework that integrates data‑lake storage with mainstream big‑data processing paradigms such as stream, batch, and OLAP analysis, while also supporting common machine‑learning and AI models. As lakehouse analytics mature, enterprises increasingly demand real‑time big‑data analysis on this architecture.

IDC predicts that within the next 12 months, the proportion of enterprises opting for external partners to build data‑management services will surge from 58% to 85%. Rapid data growth, heightened data‑management needs, and rising complexity and development costs are driving firms toward integrated lakehouse management solutions, with multi‑model data management and real‑time capabilities identified as key evolution directions.

The MarketScape evaluated 13 typical real‑time lakehouse vendors in China across capability and strategic performance, covering internet, cloud, and big‑data providers. Alibaba Cloud was highlighted as a leader.

The report notes Apache Paimon as a next‑generation real‑time lakehouse format that supports stream‑batch processing, contributed by Alibaba Cloud and shared with the open‑source community. Combined with Flink components, it builds a lake‑format + LSM architecture optimized for stream updates and offers tighter integration with Flink and Spark, SLA guarantees of 1–5 minutes, and balanced read‑write amplification.

In the AI‑for‑Lakehouse space, Alibaba Cloud provides intelligent data layout, resource usage, execution engine, query planning, resource reuse, and Copilot features. In the Lakehouse‑for‑AI context, it enables optimized management of diverse data, such as high‑throughput offline processing, low‑latency online services, low‑resource fine‑tuning and prompting for training data, and low‑carbon training for massive pre‑training datasets.

Alibaba Cloud delivers an open‑storage‑based lakehouse multi‑engine collaborative big‑data AI solution, offering unified metadata management, a single lake‑table format, and distributed data management. It integrates with major big‑data compute products like real‑time Flink, EMR, EMR Serverless Spark, EMR Serverless StarRocks, MaxCompute, and Hologres, providing lower cost, end‑to‑end real‑time flow, data updatability, full‑link traceability, and minute‑level data freshness for enterprises.

About IDC MarketScape: the vendor assessment model provides an overview of ICT vendor competitiveness in a specific market, using rigorous qualitative and quantitative scoring methods to illustrate each vendor’s position. It offers a clear framework for comparing products, services, capabilities, strategies, and market success factors, giving technology buyers a 360‑degree evaluation of current or potential vendors.