Alluxio Data Orchestration and Cache Acceleration in China Unicom: Use Cases and Performance Gains

Speaker Zhang Ce, a Big Data Engineer at China Unicom and PMC member of the Alluxio community, introduces Alluxio’s role in data orchestration and cache acceleration across four scenarios: cache acceleration, compute‑storage separation, mixed‑load workloads, and lightweight analysis.

In the cache‑acceleration scenario, Alluxio is used to write intermediate results (Sink) directly to its memory layer and read them as the next job’s source, eliminating disk I/O, improving pipeline stability and speed; it also replaces Spark Cache for shared intermediate data, offering predictable linear performance and configurable replication.

For compute‑storage separation, Alluxio mounts remote HDFS paths, enabling a unified namespace across clusters, while employing RocksDB + Raft HA for metadata and HDD storage for intermediate data, thus overcoming network latency and fragmentation issues.

In mixed‑load workloads, Alluxio’s local cache isolates Presto from Spark’s system cache, and Alluxio Fuse mounts distributed files as local files, allowing seamless integration of Spark‑ETL with TensorFlow training.

Lightweight analysis is achieved by combining Presto with Alluxio (and Iceberg) to provide a SQL‑only analytics stack that requires only two components per node, supporting both private and public data sharing.

Performance measurements show up to 70‑fold business scale growth, 50 % query‑time improvement for Presto, and an 83 % reduction in storage expansion cost, demonstrating Alluxio’s impact on stability, scalability, and cost efficiency.