Unlocking Efficient LLM Inference: Insights from China’s Cloud Computing Conference

The 5th China Cloud Computing Infrastructure Developer Conference (CID) was held in Beijing on October 19, 2024, gathering over 300 participants and featuring more than 30 technical talks covering the latest advances in cloud infrastructure.

Efficient Large‑Model Inference on GPU

Aliyun senior computing expert Zheng Xiao emphasized that the rapid growth of AI applications makes inference cost and efficiency critical. He presented a real‑world large‑language‑model (LLM) inference deployment, detailing GPU‑specific optimizations and multi‑GPU communication tuning methods that can markedly improve throughput while reducing expenses.

Accelerating Enterprise AI with Knative Serverless

Aliyun specialist Li Peng introduced Knative, an open‑source serverless framework built on Kubernetes, and explained its capabilities such as automatic request‑driven scaling, zero‑instance scaling, gray‑release, and event‑driven processing. He highlighted Aliyun’s Knative product enhancements, including a UI console, intelligent AHPA scaling, and deep integration with EventBridge, Cloud Monitor, Arms‑Prometheus, ALB, ASM, and MSE.

Focus on business logic: developers can concentrate on application code while Knative handles scaling and resource management.

Standardization: provides a vendor‑agnostic serverless framework that eases cross‑cloud migration.

Low entry barrier: supports container images and function deployment without requiring deep Kubernetes knowledge.

Automation: automatic scaling to zero when idle saves resources, and built‑in multi‑version and gray‑release features simplify deployments.

Event‑driven model: a complete event system enables seamless integration with external services.

AMD Core & Uncore PMU Virtualization

Aliyun experts Zheng Xiang and Chen Peihong explained the principles and implementation of Core&Uncore PMU virtualization on AMD instances. The solution allows performance tools such as perf and AMDuProf to run inside virtual machines, exposing memory‑bandwidth, LLC, and DMA metrics, thereby narrowing the gap between VM and bare‑metal performance monitoring.

Full‑Link GPU Management with CDI in Kubernetes/KataContainers

Aliyun senior engineer Wu Chao and Ant Group engineer Li Yanan demonstrated how the Container Device Interface (CDI) can be used to expose GPUs to Kata Containers, achieving standardized, end‑to‑end GPU lifecycle management. This approach simplifies resource allocation for AI/ML workloads and enables seamless migration across environments.

Overall, the conference showcased cutting‑edge cloud infrastructure advances—serverless AI inference, GPU resource optimization, and hardware‑level performance visibility—indicating a rapid shift toward more efficient, cost‑effective AI services in China’s cloud market.