Hot Chips 2026: HBM’s Shift to Integrated Base‑Die and 3D‑DRAM’s Alternative Roadmap

HBM at the Architectural Inflection Point

At Hot Chips 2026, Samsung, SK Hynix and Micron presented their latest HBM results and emphasized that traditional HBM development is approaching physical limits, marking a critical architectural inflection point for the industry.

Why HBM Needs Evolution

Micron highlighted the widening "scissor gap" between AI compute power and memory bandwidth, making HBM an unavoidable bottleneck. SK Hynix pointed out physical constraints such as stacking height‑induced heat, TSV area, and package stress, which narrow the path of simply adding more layers and density.

Samsung’s Three‑Stage HBM Roadmap

Stage 1 – "Making Room" : The memory controller is moved from the expensive xPU (GPU/TPU) die to a Base Die using a compact Die‑to‑Die (D2D) interface, saving 5‑10 % die area and translating into a 10‑20 % performance boost. Samsung’s Heat Path Block (HPB) technology is employed in this stage to address hotspot issues.

Stage 2 – "Growing Functions" : With leftover space on the Base Die, customized HBM (cHBM) becomes possible. Samsung integrates a memory‑expansion controller (for external LPDDR or HBF), a dedicated processing unit (PE) to off‑load tasks such as MoE expert parallelism, and RAS (reliability, availability, maintainability) features, turning HBM into a near‑data processor rather than a pure memory component.

Stage 3 – "zHBM" (3‑D‑zHBM) : This ultimate form eliminates the 2.5 D interposer by stacking HBM directly on the xPU die. Samsung claims zHBM can reduce power consumption by 70 % and increase bandwidth by 230 % compared with standard HBM4E, saving roughly 100 W per GPU. Realizing this requires Wafer‑on‑Wafer (WoW) and hybrid‑bonding advanced packaging techniques.

Packaging and HBF – The New Competitive Focus

Packaging technology is now as crucial as chip design. SK Hynix notes that competition has shifted from pure DRAM performance to advanced packaging. Its MR‑MUF (mass‑reflow under‑fill) outperforms TC‑NCF (thermal‑compression non‑conductive film) in thermal resistance and volume production efficiency. Hybrid bonding is identified as essential for stacks exceeding 20 layers, eliminating bump gaps and dramatically lowering thermal resistance and stack height.

The High‑Bandwidth Flash (HBF) concept is positioned not as an HBM replacement but as an extension via the Base‑Die D2D interface, serving as a large‑capacity, low‑cost KV cache or MoE parameter store.

3D‑DRAM as an Alternative – d‑Matrix Raptor

d‑Matrix introduced its 3D‑DRAM Raptor chip, which stacks a compute logic die directly on a DRAM die. This architecture creates ultra‑short vertical interconnects that achieve approximately 100 TB/s bandwidth with an I/O energy of 0.37 pJ/bit—far superior to HBM’s 2‑5 pJ/bit range. In generative‑AI inference scenarios, Raptor’s throughput is 4.71× that of comparable HBM systems.

The trade‑off is limited capacity (4 GB per die, 32 GB per card) and new thermal and power challenges caused by placing active logic on DRAM.

Conclusion

Hot Chips 2026 reveals that HBM is evolving from a standard memory component to a semi‑custom, functional compute‑near‑memory block built with advanced logic processes. Simultaneously, advanced packaging, hybrid bonding, and alternative 3D‑DRAM solutions define the next competitive frontier for high‑performance AI inference.