How Alibaba Cloud’s ODPS Upgrade Redefines Big Data Processing and AI Integration

ODPS Upgrade: Integrated Storage, Scheduling, and Metadata

At the 2022 Cloud Expo, Alibaba Cloud Vice President Jia Yangqing announced that ODPS has been upgraded to a unified big‑data platform to meet diverse data‑computing needs. The new ODPS supports large‑scale batch processing, real‑time analytics, streaming computation, and machine learning, and can schedule more than 100,000 servers in parallel.

The platform’s integrated architecture combines storage, scheduling, and metadata management, enabling efficient processing for scenarios such as real‑time traffic data for smart city optimization, AI‑assisted scientific research, and large‑scale risk actuarial calculations in insurance.

Since its inception in 2009, ODPS has evolved from single‑cluster processing to a financially‑grade, cloud‑native security model. In 2017, its massive batch engine MaxCompute was released as an independent product, and in 2019 the interactive engine Hologres was launched to address real‑time analytical demands.

In the recent TPC‑H 30,000 GB benchmark, ODPS‑Hologres set a world record with a score exceeding 27.86 million, surpassing the runner‑up by 23%. ODPS‑MaxCompute has also maintained the top performance‑to‑cost ratio for six consecutive years in the TPCx‑BB 100 TB test.

Big Data and AI Converge: Xpeng AI Training Accelerated 170×

Jia emphasized that big‑data platforms and AI platforms are increasingly merging. Whether for large‑model production, AI for Science, or traditional search and advertising, efficient data flow and analysis are essential.

In collaboration with Xpeng Motors, Alibaba Cloud built an integrated big‑data‑AI platform that streamlines the entire pipeline from massive road‑capture data ingestion to AI model training and inference. Leveraging hardware‑software co‑optimization, Xpeng’s autonomous‑driving core model training speed increased by nearly 170 times, with a 30 % performance gain over open‑source solutions.

Since launching the “Alibaba Lingjie” big‑data + AI product suite, Alibaba Cloud has expanded data‑intelligence applications across industries. The PAI machine‑learning platform strengthens AI engineering capabilities, supporting diverse deep‑learning frameworks and hardware on a flexible, cloud‑native AI platform. Additionally, the DAMO Academy’s ModelScope and OpenMind services further democratize AI.