Design and Optimization Practices for Intelligent Computing Platforms in the Era of Large Models

This article, originally part of the "Architect Summit: Large Model Applications and Practices Collection," discusses the design and optimization of intelligent computing (智算) platforms in the era of large AI models.

New characteristics of large‑model era platforms: Differences between small and large models in training duration, cost, and engineering complexity; the need to address infrastructure, scheduling, application, and operation challenges; requirements such as supporting heterogeneous chips, optimizing storage I/O, and building high‑performance networks.

Problems the platform must solve: Infrastructure layer issues like chip, firmware, and driver compatibility; scheduling layer challenges of efficiently allocating massive heterogeneous compute resources; application layer demands for training and inference acceleration and fault tolerance; operation layer goals to improve failure handling and capacity management.

Technical practices in large‑model scenarios: Infrastructure layer – compatibility between domestic and NV GPUs, mixed‑chip usage, high‑performance storage solutions; scheduling layer – improving per‑GPU utilization, GPU virtualization, resource management and scheduling logic; application layer – AI‑accelerated training/inference, training fault tolerance, Flash Checkpoint; operation layer – fault handling, capacity management, performance tuning.

Future considerations for platform development: Simplify lower‑level complexity to make GPU usage more convenient; act as a bridge connecting heterogeneous resources and supporting AI platforms; anticipate increasing difficulty of pre‑training, diversified fine‑tuning, and potential growth in model inference.

Overall, the article highlights that, with continuous technical innovation and optimization, intelligent computing platforms can significantly improve performance and stability for large‑model workloads, thereby accelerating AI technology development.

Additional reading links provide further insights into related topics such as InfiniBand, RoCE, high‑performance networking, CPU architectures, and AI‑focused hardware.