Next‑Gen MaxCompute: Full‑Modal Engine and Agentic Upgrade Overview

MaxCompute (formerly ODPS) has entered a transformation phase that expands its original structured‑data, SQL‑centric capabilities into two upgrade tracks: Data+AI and Agentic. The Data+AI track unifies model object management, heterogeneous resource scheduling, full‑modal storage, and AI Function as the entry point for data‑model interaction, enabling seamless data preparation, processing, and inference.

Data+AI Upgrade

Model layer: Models from public sources (e.g., Bailei commercial models, Qwen3, DeepSeek) and custom models (OSS‑hosted, PAI EAS endpoints, traditional ML models) are registered as MaxCompute model objects with unified metadata, permission, versioning, and CRUD operations, callable from SQL or MaxFrame AI Functions.

Compute layer: Three quota types—CU (CPU), GU (GPU), and Inference (token‑based)—serve different workloads: CPU for small models and testing, GPU for high‑throughput large‑model jobs, and token‑based inference for commercial large models. Resources are managed serverlessly, abstracting underlying hardware details.

Storage layer: Introduces a Blob data type allowing binary objects (images, audio, video, documents) to be stored directly in MaxCompute tables. This supports AI training data management, multimodal pipelines, caching, and lake ingestion, reducing data movement across stages.

AI Function: Provides SQL functions (AI_GENERATE, AI_CLASSIFY, AI_EXTRACT, etc.) and MaxFrame SDK interfaces to embed model inference in ETL workflows. It supports automatic concurrency splitting, worker sleep, rate limiting, and distributed execution, while exposing resource quotas and token billing within the execution chain. Results remain nondeterministic due to model and prompt variability.

Full‑Modal Data Pipeline

The pipeline demonstrates end‑to‑end processing of multimodal data, illustrated by two cases:

Autonomous‑driving data line: vehicle telemetry, manual annotations, video files are ingested via DataHub/Kafka, integrated in DataWorks + MaxCompute, pre‑processed, temporally aligned, and labeled by large models. Results flow to OSS sample stores, Hologres, and vector databases for downstream PAI model training.

Ego‑centric pipeline for embodied intelligence: raw human‑hand videos undergo CPU‑based splitting and frame extraction, GPU‑based depth estimation and 3D pose estimation, and token‑based large‑model actions segmentation and semantic labeling, producing the LeRobot dataset.

Engineering safeguards include per‑segment failure logging without halting the whole line, GPU memory reuse (HaWoR), OSS mount conflict avoidance, and independent MaxCompute tables for each stage to enable checkpoint‑based recovery. Reported metrics show processing of over 1 million images and 5 billion tokens, more than 2× performance improvement, and a reduction of the raw‑to‑train‑sample cycle from weeks to days.

Agentic Upgrade

Agentic capabilities extend MaxCompute access from human users to enterprise‑built agents, platform agents, and third‑party agents. The architecture consists of three layers:

Upper layer: Agent entry points and tools such as MaxAgent and the AI Data Exploration client.

Middle layer: MCP, Skills, and Metadata + Semantic APIs that wrap low‑level OpenAPI, StorageAPI, CatalogAPI into agent‑friendly methods, enhancing NL2SQL and “talk‑to‑data” interactions.

Lower layer: Core MaxCompute storage, compute engine, heterogeneous resources, model registry, and governance features.

To address repetitive queries and automated writes, the platform offers AutoMV (automatic materialized views), Delta Table, and Time Travel for cost‑effective query handling and safe data rollback. MaxAgent supports four scenario categories—daily data‑dev operations, mobile resource administration, cost‑reduction & efficiency, and intelligent Q&A—integrating job diagnostics, view recommendations, auto‑cluster optimization, quota routing, cost attribution, storage observation, NL2SQL, contextual follow‑up, and cross‑domain “query + diagnose” workflows, with optional DingTalk, Feishu, or WeChat bot channels.

MCP exposes granular capabilities: metadata enumeration (projects, schemas, tables, partitions), cost‑estimated SQL execution, instance status queries, access checks, and data‑management actions (create/insert tables). Skills extend these with SQL generation, MaxFrame coding, information‑schema queries, and quota management. Production constraints such as AK/SK, STS, credential URIs, read‑only execution, cost‑approval, and lifecycle policies are enforced at the MCP level.

The AI Data Exploration client consolidates catalog browsing, SQL editing, result visualization, and Data Agent in a single workspace. Although not yet a commercial service, it is released as an open‑source project for community extension and integration.

Overall, the full‑modal compute and Agentic upgrades converge on a unified data foundation that handles structured and unstructured assets, encapsulates query, operation, resource, and cost governance into agent‑usable interfaces, and retains the underlying security, permission, and recovery mechanisms of MaxCompute.