Alluxio Deployment at Ant Group: Stability Building, Performance Optimization, and Scale‑up for Large‑Scale Model Training

Ant Group adopted Alluxio to overcome three core challenges in large‑scale GPU model training: storage I/O bottlenecks, single‑node capacity limits, and network latency, seeking a solution that combines high throughput, concurrency, and low latency.

Stability Building focuses on two areas: worker‑register follower and master migration. By registering each worker with all masters and maintaining a heartbeat between primary and workers, the fail‑over (FO) time can be reduced to under 30 seconds, minimizing user‑visible errors. Master migration issues are solved by dynamically updating workers with the current master set via a primary‑worker heartbeat.

Performance Optimization introduces a follower‑read‑only mode. After the initial metadata warm‑up, standby masters serve read‑only requests without affecting Raft journal entries, allowing three‑fold throughput improvements and better utilization of standby resources for read‑heavy workloads.

Scale‑up is achieved through horizontal expansion and metadata sharding. By partitioning metadata across multiple clusters and routing client requests via a proxy that hashes keys to specific shards, the system can support billions of files, alleviate memory pressure on block and file masters, and increase overall QPS and throughput.

The combined optimizations—stability, performance, and scaling—enable Ant Group to support ever‑growing model training workloads with reduced fail‑over times, higher throughput, and the ability to handle massive data volumes.