Why AI Servers Are Driving a Storage Revolution: HBM, CXL, and Beyond

Traditional single‑node servers typically cost under $10,000, whereas a DGX H100 AI server equipped with eight H100 accelerator cards can exceed $400,000 (about 3 million RMB), highlighting the massive value shift toward AI‑focused hardware.

AI workloads such as large‑scale language models and graphics rendering demand massive memory bandwidth, creating a bottleneck known as the “memory wall.” As CPU performance outpaces DRAM bandwidth, high‑bandwidth memory (HBM) has become a key solution. HBM, built on 3‑D stacking, offers over 1 TB/s of bandwidth (HBM3E) with lower power consumption, making it ideal for HPC and AI servers.

Market data from Yole Group shows that in 2023 SK Hynix captured 55% of HBM revenue, followed by Samsung (41%) and Micron (3%). Product comparisons reveal Micron’s strength in low power, Samsung’s advantage in tighter stack spacing for higher layer counts, and SK Hynix’s MR‑MUF technology for superior thermal performance.

Looking ahead, Micron plans to launch a 36 GB 12‑High HBM3E in 2025, followed by a 36 GB 12‑High HBM4 in 2026, a 48 GB 16‑High HBM4 in 2027, and a bandwidth‑boosted HBM4E (>2 TB/s) in 2028, illustrating an aggressive product roadmap.

Beyond memory, AI server storage requirements are driving new interconnect standards. Compute Express Link (CXL) introduces DRAM pooling, reducing data‑center construction costs and increasing DRAM utilization. Multiplexer Combined Ranks (MCR/MDIMM) promise up to double the bandwidth of DDR5, with AMD and Intel backing open‑standard developments. PCIe 5.0 provides a next‑generation high‑speed serial bus to support these advances.

Chinese fabless company Montage Technology (澜起科技) released its MCR control chips (MRCD/MDB) in 2022 and offers a portfolio that includes memory‑interface chips, PCIe retimers, MXC and CKD chips, as well as server platforms featuring CPUs and hybrid secure memory modules. Their roadmap emphasizes continued DDR5 iteration, scaling of retimer and PCIe 6.0 products, and expanding high‑performance interconnect solutions.

In the DDR5 era, SPD EEPROMs become critical for module identification and configuration across LR‑DIMM, RDIMM, UDIMM, and SODIMM form factors. As DDR5 adoption grows, the market for SPD EEPROMs is expected to expand significantly.