Why HBM3E Is Set to Power the Next AI Server Boom

HBM (High Bandwidth Memory) stacks multiple DRAM dies vertically using TSV (through‑silicon vias) to connect to a logic die, achieving high bandwidth, low power consumption, and a compact form factor, making it the mainstream solution for GPU memory in high‑performance AI servers.

The latest iteration, HBM3E, was introduced by SK Hynix and offers up to 8 Gbps transfer speed and 16 GB (planned 24 GB) capacity, with mass production slated for 2024 and integration in Nvidia's 2023 H200 accelerator.

AI server shipments rose from 860,000 units in 2022 to an expected >2 million units by 2026 (CAGR ≈ 29%). This surge drives explosive HBM demand; market analysts estimate the HBM market will reach roughly $15 billion in 2025, growing at >50% annually as server memory capacity increases.

According to TrendForce, the 2023 HBM supplier market share is roughly SK Hynix 53%, Samsung 38%, and Micron 9%. These vendors focus on DRAM‑die production and stacking technology upgrades, with SK Hynix leading the early supply of HBM3E to Nvidia.

Two primary packaging technologies dominate HBM integration:

CoWoS : Combines DRAM dies and GPU on a silicon interposer, shortening interconnect length for faster data transfer. It is the mainstream solution used in Nvidia A100, GH200 and similar accelerators.

TSV : Creates thousands of vertical connections through the wafer thickness, enabling multi‑die stacking. Only the bottom die connects to the external controller, while internal dies communicate via TSV.

HBM evolution timeline:

HBM1 (2014, AMD + SK Hynix): 4‑die stack, 128 GB/s bandwidth, 4 GB capacity.

HBM2 (2016/2018): 4‑8‑die, 256 GB/s bandwidth, 2.4 Gbps per pin, 8 GB capacity.

HBM2E (2018/2020): 3.6 Gbps per pin, 16 GB capacity.

HBM3 (2020/2022): 6.4 Gbps per pin, up to 819 GB/s bandwidth, 16 GB capacity.

HBM3E (2024): 8 Gbps per pin, 24 GB capacity, slated for large‑scale production.

HBM has been deployed in AI servers since 2016, starting with Nvidia NVP100 (HBM2), followed by V100 (HBM2), A100 (HBM2), H100 (HBM2e/HBM3), and the latest H200 (HBM3E), providing higher speed and capacity for demanding workloads.

Key suppliers SK Hynix, Samsung, and Micron compete in DRAM‑die manufacturing and stacking processes; SK Hynix’s early partnership with AMD and Nvidia gives it a market lead, while Samsung supplies other cloud providers and Micron holds a smaller share.

Related reading (titles only):

算力竞赛，开启AI芯片、光模块和光芯片需求

AI算力租赁行业深度研究（2023）

大模型算力：AI服务器行业（2023）

GPU技术篇 – 全球竞争格局与未来发展

2023年GPU显卡技术词条报告

GPU/CPU散热工艺的发展与路径演绎

新型GPU云桌面发展白皮书

ChatGPT对GPU算力的需求测算与分析