Alibaba Cloud ODPS Upgrade: AI-Native Multimodal Big Data Infrastructure for Agents

ODPS Strategic Upgrade for Agentic AI Era

At the 2026 Yunqi Conference, Alibaba Cloud announced a comprehensive upgrade of its self-developed big data platform ODPS, repositioning it as an AI-native, multimodal data infrastructure for the Agentic AI era. The upgrade spans three core products: ODPS-MaxCompute, ODPS-Hologres, and ODPS-DataWorks, each redesigned around Agent-native architecture and multimodal compute capabilities.

ODPS-MaxCompute: Everything Is Computable

MaxCompute lead Zhang Zhiguo presented the core release. As multimodal data becomes mainstream in GenAI, the processing unit shifts from rows to video segments and audio-visual groups, which traditional ETL cannot handle. MaxCompute underwent a full-stack rebuild across storage, compute, engine, and interaction.

SQL AI: In-Warehouse Vectorization and Retrieval

The most visible change is that a single SQL statement can now handle end-to-end multimodal processing. Previously, data had to move across systems for embedding, indexing, and retrieval — long chains, high cost, complex ops. MaxCompute SQL AI now adds EMBEDDING and VECTOR_SEARCH capabilities, closing the loop from vectorization to retrieval inside the warehouse with zero resident services. This marks the fourth evolutionary step: 2023 Data+AI, 2024 Object Table for unstructured data in SQL, 2025 AI Function for direct model invocation, and 2026 heterogeneous compute support with vector search closure and full multimodal processing.

Case studies demonstrate impact:

Zhiyi Technology migrated million-scale product image vector matching from a dedicated retrieval cluster to MaxCompute SQL AI: CPU consumption down 73%, memory down 93%, retrieval accelerated 5.86×.

Qwen APP RAG corpus evaluation on 24M records and 1.2M queries: full-pipeline evaluation time reduced from 6h15m to 32m, cost cut 80%.

Multimodal Storage and Heterogeneous Compute

Underlying these scenarios is a new multimodal storage and heterogeneous compute system:

Storage : Blob large-object type uses reference-entity separation; text auto-compression saves ~5× space. Object Table manages unstructured data on OSS with caching and query optimization. Together they enable mixed multi-column storage of text, image, audio, video, 3D, and sensor data.

Compute : Three quota types — CU Quota (general CPU with AutoScaling), GU Quota (serverless GPU, elastic per-job thousand-card scheduling, auto bad-card replacement), and Inference Quota (integrated with Alibaba Cloud Bailian LLM, token-metered billing). Resources switch on demand, enabling true pay-as-you-go Data+AI fusion compute. Deep integration with Bailian commercial LLMs via SQL + MaxFrame dual engines, with TPM/RPM rate-limiting, auto concurrency splitting and backoff, pushes large-scale inference to the limit under same quotas.

MaxFrame: Distributed AI Compute Engine

MaxFrame is upgraded to a distributed AI compute engine with DPE heterogeneous resource unified scheduling supporting 100k CU cluster capacity, mixing CPU/GPU/Token resources and auto-matching per operator.

Qiongche Intelligence used UMI hand-held gripper to collect fisheye video and 6DoF pose data; via pose reconstruction, semantic annotation, LeRobot dataset generation, leveraging MaxFrame AI Function and serverless heterogeneous compute, data processing throughput increased 10×+, elastic compute peaked at 100k+ CU, model training efficiency up ~50%, fully unattended.

Yiwu Commodity City used MaxFrame AI Function to understand 100M+ product images/videos; million-CU elastic scaling doubled performance vs self-built, integrated Qwen and DeepSeek for one-click multilingual product description generation.

MaxFrame also releases industry Skills, encapsulating senior engineers' experience into reusable AI Skills, drastically lowering industry data pipeline setup barriers.

Agentic Interaction Suite

MaxCompute launches a full-scenario Agentic suite: AI Query (natural language querying), AI Coding (code assistance), MaxAgent (full-link intelligent ops). MaxAgent executes as the user without modifying data, covering six ops scenarios — job diagnosis, cost optimization, permission audit — reducing 30-minute manual troubleshooting to 30 seconds. The MCP Server wraps platform APIs as Agent-callable tools, enabling third-party Agent plug-and-play. A platform-hosted semantic layer materializes metadata, lineage, business terms, and metric definitions into SemanticSpec, giving Agents context on demand to reduce trial-and-error; an ops semantic package lets Agents go from problem to action directly.

ODPS-Hologres 5.0: From Real-Time Data to Reliable Action

Hologres lead Jiang Weihua noted OLAP users are shifting from humans to Agents. Agents run continuously, autonomously combine context, and feed results back into the next reasoning round. This brings lower barriers, continuous operation, and closed-loop action, but also four new challenges: real-time complete data, understandable semantics, trustworthy controllable actions, and adaptive execution for unpredictable queries. Hologres 5.0 upgrades to an Agent-native multimodal real-time data warehouse addressing each.

Data Sandbox: Safe Experimentation

The biggest obstacle for Agents entering enterprise data pipelines is trust — Agents could only read, not write, because a wrong write impacts production. Hologres 5.0 introduces Data Sandbox , creating an isolated writable workspace from the production instance. Agents can freely execute CRUD and DDL experiments; after validation and approval, changes merge back to the source without affecting production. A live demo showed an Agent completing a four-step loop for a cross-channel GMV drop analysis — identify low-conversion channel, experiment with metric definition changes, replay historical data to compare old vs new definitions, generate approved job for merge — turning multi-round manual investigation into a single automated closure.

Semantic Understanding: Graph Query and Semantic View

Trust also requires Agents to truly understand business data meaning. Hologres 5.0 adds graph query capability via AGE extension, allowing SQL to call a Cypher subset and combine with regular SQL. Relationship chains like "customer → places order → order → contains → product" are directly expressible in SQL, supporting knowledge graph construction and Agent decision-making. Simultaneously, Semantic View builds a business semantic layer inside the warehouse: declare table relationships and metric definitions, and the engine auto-rewrites multi-table JOINs while guaranteeing aggregation semantics, so humans, BI, and AI share the same business language, improving LLM data retrieval accuracy.

User Defined Agent and Hologres Agent

For Agent definition and execution, Hologres 5.0 introduces User Defined Agent : users define AI processing logic in Python and embed it in data pipelines for continuous automatic runs. Using DataFrame to trigger AI inference per row, combined with Dynamic Table incremental materialization, only new rows are processed on source inserts — no full recomputation. The same Agent logic supports both batch and streaming. Example: product review AI scoring — each new review yields a score and rationale in the scoring table within 1 minute.

Hologres also releases Hologres Agent , a console-built intelligent assistant supporting natural language for product consultation, instance ops, warehouse development, and data analysis. On this foundation, three specialized Agents launch: Ops Agent, Tuning Agent, and Data Agent, sharing real-time data, Skills, and security boundaries.

Three Trust Pillars: Rememberable, Tryable, Observable

Around "rememberable, tryable, observable", Hologres 5.0 builds a complete trust foundation for Agents:

Data Sandbox makes Agents tryable — isolated experiments with approval merge.

Long-term Memory Service makes Agents rememberable — scores 96.82 on LOCOMO benchmark (SOTA), token savings >90%.

HoloScope makes Agents observable — 100+ metric ingestion and full-session replay evaluation.

All three are built on the Hologres base, serving any general Agent. Performance-wise, Hologres continues to set records: VectorDBBench vector retrieval leads the industry; TPC-H 3TB world #1, providing solid performance guarantees for unpredictable Agent query patterns.

ODPS-DataWorks: Bringing Agents into Enterprise Data Production Lines

DataWorks lead Tian Qixi observed enterprise data processing is shifting from "people using tools" to "Agents running autonomously", but Agents struggle with same-name-different-meaning, same-meaning-different-name fields, and generating SQL that looks correct but picks wrong tables — "Agents are smart but lack business common sense." To solve this, DataWorks releases the Enterprise Semantic Knowledge Graph (Context Graph) , auto-generated by scanning and inferring from lakehouse/warehouse data assets — no manual ontology/modeling required. It covers six semantic layers: metadata enrichment, metrics & terminology, relationships & lineage, scheduling & quality, intelligent learning, and organizational knowledge. It updates incrementally with asset changes and is shared by all Agents. Before Context Graph, cross-system analysis required serial confirmation across systems; after, a single question auto-injects context. Business QA accuracy reaches 93.24%, SQL generation consistency 99%, model cost lower, with Qwen3.8-flash approaching flagship model performance.

DataWorks accumulates 10+ years of data development practice: 40M+ daily scheduled tasks, EB-scale data management, 15+ official expert suites, 80+ skills covering integration, development, ops, governance, analysis end-to-end, cloud-hosted 7×24 online, integrated with DingTalk, Feishu, WeCom. The new ADA (AI-Native Big Data Service) enables multi-Agent collaborative orchestration and cross-engine autonomous task delivery across offline, real-time, streaming, and AI compute scenarios.

Three Guardrails for Trusted Adoption

"Trustworthy" is the final barrier for enterprise Agent adoption. DataWorks builds three defenses:

Data Security : identity, fine-grained permissions, security sandbox, sensitive data protection.

Behavior Audit : full-link observability, Agent data lineage and watermark traceability.

Cost Control : prevent credit runaway.

Multiple Agent engines available, open MCP, OpenAPI, 200+ Skills, and six extension capabilities. Proactive intelligent ops inspection shifts from passive human patrols to Agent autonomous discovery, root-cause, repair, and daily reports. DataWorks Data Agent ranked Leader in 2026 IDC China Data Agent Vendor Assessment.

From Cloud Infrastructure to AI-Native Multimodal Big Data Infrastructure

Over 16 years, ODPS evolved from supporting Alibaba internal businesses to becoming universal cloud infrastructure, and now fully reconstructed for the Agentic AI era — each evolution tracking the technology pulse. Alibaba Cloud's Agent-native, multimodal intelligent data platform continuously lowers AI adoption barriers, empowering thousands of industries to unlock data value in the Agentic AI era, moving from real-time data to reliable action.