AI Server Market 2024: Growth Trends, Types, and Key Challenges

Based on the analysis "2024 AI Server and AI PC Trends," data centers can be categorized into three types: cloud data centers serving diverse applications, intelligent computing centers built on AI‑specific chips for industrial AI workloads, and supercomputing centers focused on scientific and engineering calculations.

Global AI server shipments exceeded 1.208 million units in 2023, a year‑over‑year increase of 37.7 %. Trendforce forecasts 1.672 million units for 2024, a 38.4 % rise. TSMC indicated that AI demand will continue to grow at a compound annual growth rate of 50 % through 2028, suggesting sustained expansion of AI server demand.

In China, the AI server market reached US$9.1 billion in 2023, with intelligent compute capacity projected at 414.1 EFLOPS—a 59.3 % year‑over‑year increase. The sector is expected to maintain a 33.9 % CAGR from 2022 to 2027.

AI servers are divided into training and inference categories. Training servers require large storage, high bandwidth, and massive compute power, typically employing 8‑GPU designs. Inference servers have lower resource demands and may use GPUs, NPUs, CPUs, or PCIe‑based AI accelerators depending on the workload.

Server form factors have evolved from general‑purpose to cloud, edge, and finally AI‑optimized servers. The core of AI servers combines CPUs and GPUs, and rack‑level solutions are becoming the dominant deployment model.

The industry faces three major challenges: (1) ensuring the quality and reliability of advanced multi‑layer HDI GPU cards; (2) developing high‑speed connectors (56G‑112G‑224G) and dense back‑plane solutions to meet bandwidth demands; and (3) addressing thermal limits of high‑performance AI chips, which now exceed 1000 W, making liquid cooling a necessary trend.