How Cloud‑Native AI Boosts Resource Efficiency with PaddleFlow

1. Introduction to Cloud‑Native AI

Cloud‑native AI applies cloud‑native technologies to AI workloads, enabling elastic scaling and seamless migration across public, private, and hybrid clouds. It abstracts underlying infrastructure, allowing developers to focus on rapid deployment and on‑demand scaling.

Key pain points addressed include resource efficiency (utilization, heterogeneous chip scheduling, virtualization, network) and engineering efficiency (model deployment, training/inference speed, image startup).

Uneven resource allocation leads to high‑priority tasks failing to run efficiently.

Resource fragmentation prevents tasks from using idle cluster capacity.

Low GPU utilization reduces overall throughput.

Complex distributed training orchestration and large AI images slow down engineering workflows.

Cloud‑native AI mitigates these issues through containerization, unified resource management, and advanced scheduling.

2. Resource Scheduling in Cloud‑Native AI

The platform provides a multi‑layer architecture:

Resource Management Layer : Heterogeneous chip management, GPU container virtualization (dual‑engine), remoteGPU, Kunlun chip virtualization, high‑performance RDMA and storage access.

AI Scheduling Layer : Multiple scheduling algorithms integrated to create high‑performance execution environments.

AI Task Management Layer : Operators for distributed training, workflow engine, and acceleration for training, inference, image, and data.

A global resource view aggregates CPUs, memory, GPUs, GPU memory, and custom resources. Administrators define resource quotas and queues.

Two queue types exist:

Overcommit (超发) Queue : Allows tasks to exceed quota when tagged; excess tasks are preempted first.

Non‑overcommit Queue : Rejects new tasks when resources are insufficient.

Scheduling actions include enqueue, allocate, recycle, preempt, and backfill, operating on PodGroup units (strongly related Pods). Core algorithms:

Gang Scheduling and Gang Preemption ensure all Pods of a group are scheduled together.

Bin‑packing reduces resource fragmentation by filling nodes before using new ones.

GPU Offline Mixed Scheduling balances online latency‑sensitive and offline throughput‑oriented workloads.

GPU Topology‑Aware Scheduling places GPUs with maximal NVLink bandwidth on the same NUMA node.

Tor‑Aware Scheduling co‑locates pods of the same training unit within the same Tor switch to minimize network congestion in large‑scale training.

3. AI Workflow Engine – PaddleFlow

PaddleFlow bridges AI engineers and cloud‑native infrastructure, offering a unified compute and storage interface, support for major AI frameworks, and a DAG‑based workflow engine.

Architecture consists of two parts:

AI Job Scheduling System (green): Provides built‑in deep‑learning and traditional ML engines, a pipeline core for DAG execution, and multiple user interfaces (CLI, SDK, Web UI).

Resource Management System (red): Abstracts compute and storage resources, reuses the scheduling algorithms described above, and integrates PaddleFlowFS for high‑performance data access.

Key features:

Python DSL and YAML for workflow definition.

Checkpoint‑resume for failed tasks.

Automatic caching of intermediate data.

Zero‑intrusion I/O management for data archiving and reproducibility.

Rich DAG capabilities, including sub‑DAGs.

Hierarchical scheduling with elastic quota and multi‑tenant queues.

Location‑aware scheduling to co‑locate compute and storage.

PaddleFlowFS provides a Fuse client, SDK, and CSI driver, abstracting various remote storage systems (HDFS, S3‑compatible) via a VFS layer and a two‑level cache, delivering 5‑10× faster first‑read latency and >30% improvement on cached reads.

PaddleFlow is open‑source on GitHub and can be deployed with a one‑click entry on Baidu Cloud Container Engine (CCE), offering a stable, high‑availability environment.