AI Server Market Overview and Technical Architecture

The piece titled "Artificial Intelligence Ignites Compute Demand, AI Server Market Dissection" introduces the composition and market dynamics of AI servers, covering hardware components, the impact of AIGC on server demand, training and inference workloads, and the overall market opportunities and competitive landscape.

Server hardware typically includes processors, memory, chipsets, I/O (RAID cards, network cards, HBA cards), storage devices, power supplies, and cooling systems, with a typical cost breakdown of roughly 50% CPU/chipset, 15% memory, 10% external storage, and 25% other components.

The logical architecture mirrors that of a standard computer, emphasizing the CPU and memory as the core elements for logical operations and data management.

Firmware components consist of BIOS or UEFI, BMC, CMOS, and operating systems supporting both 32‑bit and 64‑bit architectures.

Market data shows that global server shipments grew 6% in 2022 to 13.8 million units, with revenue rising 17% to $111.7 billion; China's server market expanded from $18.2 billion in 2019 to $27.34 billion in 2022 (CAGR 14.5%), projected to reach $30.8 billion in 2023.

According to IDC, the Chinese server market is led by Inspur and H3C, with Supercomputing (超聚变) climbing to third place; other notable players include Dell, Lenovo, and ZTE, while market share shifts reflect strong growth for Supercomputing and ZTE.

The AI server ecosystem is described in three layers: an upstream foundation of pretrained models, a middle layer of verticalized, scenario‑specific models and tools, and an application layer delivering text, image, and audio‑video generation services to end users.

Heterogeneous computing combines CPUs and GPUs via PCIe, where CPUs handle complex serial tasks and GPUs accelerate data‑intensive parallel workloads, enabling optimal performance for AI computations.

AI servers are expected to grow rapidly, with TrendForce forecasting a CAGR of 10.8% (2022‑2026) and IDC projecting the Chinese AI server market to increase from $5.7 billion in 2021 to $10.9 billion by 2025 (CAGR 17.5%).

An example configuration, the Inspur NF5688M6, features NVSwitch for GPU‑to‑GPU high‑speed interconnect, eight NVIDIA Ampere GPUs, dual Ice Lake Xeon CPUs, multiple NVMe SSDs, PCIe 4.0 slots, high‑capacity power supplies, and redundant cooling.

Core AI server components include GPUs, DRAM, SSDs, RAID cards, CPUs, NICs, PCBs, high‑speed interconnect chips, and cooling modules, with major suppliers such as Intel for CPUs and Nvidia, Cambricon, and HaiGuang for GPUs.

Globally, the top four cloud providers (Google, AWS, Meta, Microsoft) account for 66.2% of AI server procurement, while in China, ByteDance leads with a 6.2% share, followed by Tencent, Alibaba, and Baidu.