How Alibaba Cloud’s New Storage Stack Powers AI Training, Inference, and Agents

AI competition has moved from model size to real‑world business impact, driving exponential growth in data volumes—training datasets now reach hundreds of PiB, model parameters climb to tens of trillions, and KV‑Cache sizes explode in long‑context inference. These trends create new storage bottlenecks across training, inference, and emerging Agent workloads.

1. Training‑stage storage bottlenecks and CPFS

The training pipeline (data load → compute → checkpoint write → iteration) now faces shorter checkpoint intervals, massive write volumes, and mixed hot‑cold data that make all‑SSD solutions cost‑prohibitive. Alibaba Cloud responds with CPFS, a full‑stack parallel file system built on the Pangu distributed storage foundation. CPFS uses a scale‑out architecture with separate metadata and data expansion, DPU‑offloaded storage protocol stacks, and zero‑copy data paths, delivering up to hundreds of TB/s throughput and billions of IOPS. It scales linearly to hundreds of PiB and trillions of files while offering multiple storage tiers and intelligent lifecycle management to cut costs.

In a real‑world training job for the Tongyi Qianwen model on a near‑10k‑GPU cluster, CPFS reduced average model startup time by 50% and increased peak compute utilization by 30%, effectively doubling overall workload capacity.

2. Inference‑stage KV‑Cache challenges and KVCacheStore

Long‑context, multi‑turn dialogue pushes KV‑Cache sizes beyond GPU memory, making storage latency a critical factor. Four key inference requirements are identified: accelerating 1M+ context windows, preserving KV state across node failures, high‑frequency KV shuffling in P/D‑separated architectures, and sharing common prompts across requests.

KVCacheStore implements a G3.5‑level KV‑Cache storage layer, the first of its kind in public clouds. It provides 40 GB/s per compute node and can sustain millions of QPS via batch interfaces. Benchmarks show a 20% increase in cache‑hit rate and noticeable inference‑service efficiency gains.

3. Agent‑native storage with AgenticFS

Agent workloads now require isolated workspaces, per‑Agent quotas, and fine‑grained permission controls at massive scale (millions of agents, 10 k QPS mount/unmount). Traditional file systems cannot meet these elasticity and security demands.

AgenticFS introduces the AgenticSpace concept, giving each Agent an independent namespace, inode pool, capacity quota, and performance isolation. It supports dynamic creation of millions of AgenticSpaces and 10 k QPS mount‑unmount operations. Security is enforced via Access Point and RAM‑based authorization, ensuring cross‑Agent isolation.

In a production C‑end AI assistant serving millions of concurrent sessions, AgenticFS enables per‑session workspaces that survive sandbox restarts, preserving source code, artifacts, and user inputs without data loss.

4. OSS Agent and CDE Agent for intelligent storage operations

OSS Agent extends natural‑language interaction to OSS management. In a case study with Meitu, OSS Agent reduced manual bucket inspection and anomaly diagnosis by allowing users to query bucket status, capacity, and access patterns in plain language, and to visualize hot‑cold data distribution for cost optimization.

CDE Agent builds on the Qoder agent framework, integrating large‑model inference, memory, and toolchains with enterprise Netdisk’s file‑level permissions. It reuses existing AD/LDAP, DingTalk, and RAM identities, providing file‑level access control without requiring a separate security stack.

Conclusion

From training to inference to Agent deployment, Alibaba Cloud’s storage portfolio—CPFS, KVCacheStore, AgenticFS, OSS Agent, and CDE Agent—covers the entire AI data lifecycle. By accelerating storage throughput, improving cache efficiency, and delivering secure, elastic Agent workspaces, the stack turns storage speed into a decisive lever for AI compute efficiency and cost reduction.