High Bandwidth Flash (HBF): A Complete AI Storage Hierarchy Analysis

Why AI Inference Needs a New Memory Tier

Large models and multi‑agent AI workloads strain existing infrastructure; Nvidia's B200 accelerator with 192 GB HBM3e still requires frequent data swapping when running Llama 3.1 405B, hurting latency and cost, while enterprise SSDs, though capacious, exhibit 80‑150 µs latency that is unacceptable for real‑time inference.

High Bandwidth Flash (HBF) Concept

HBF is a high‑bandwidth stacked storage medium built on 3D NAND flash. By adopting HBM‑style TSV packaging, multiple NAND dies are vertically stacked with a dedicated logic die, enabling parallel access to sub‑arrays. This design leverages NAND’s low cost and high capacity while approaching HBM’s bandwidth.

Technical Principles

Sub‑array Parallel Architecture: Traditional NAND is limited by few read/write channels. HBF partitions the NAND core into many independent sub‑arrays, each with its own channel, allowing aggregate bandwidth to increase by orders of magnitude.

Logic “Brain” (Base Die): The Base Die orchestrates all NAND Core Die operations, error correction, and data scheduling, exposing a standard UCIe interface so the stack can connect to GPUs or CPUs as seamlessly as HBM.

Manufacturing reuses HBM experience: TSV vertical interconnects, TCB or hybrid bonding stack up to 16 NAND layers, and the CBA (CMOS Direct Bonded Array) process that bonds storage arrays to logic wafers to improve yield and performance.

Industry Positioning: HBM’s Complement, Not Competitor

HBF is intended as a capacity supplement to HBM. In a hybrid architecture, HBM stores hot, frequently accessed data (e.g., activation tensors), while HBF holds cold, read‑dominant data such as loaded model weights, KV‑Cache history blocks, and vector‑database shards.

Benchmark Evidence

SK Hynix’s IEEE paper on the “H3” hybrid scheme simulated an AI inference system using eight HBM stacks plus eight HBF stacks. Compared with an all‑HBM design, the mixed configuration achieved a 2.69× increase in performance‑per‑watt, demonstrating significant energy‑efficiency and cost advantages.

Supply‑Chain Perspective

Upstream: 3D NAND wafer technology (200+ layers, 5 TB prototypes from Kioxia) directly impacts HBF performance and yield.

Midstream: Chip design and packaging are the most critical barriers. SanDisk originated the HBF concept and plans a pilot line in late 2026 with commercial launch in 2027. SK Hynix leverages its HBM packaging expertise to co‑develop standards, while Samsung has announced its own HBF product roadmap.

Downstream: AI inference servers are the primary target, especially for large‑model inference, multi‑agent tasks, and ultra‑long context windows. Nvidia intends to integrate HBF into its 2026 CMX platform.

Standardization and Outlook

In August 2026, SK Hynix and SanDisk released the first HBF specification (v0.7.0) under the OCP framework, defining 8‑ or 16‑layer stacks up to 512 GB, bandwidth of 0.4‑3.0 TB/s, and a unified UCIe interface. Industry leaders such as Google and Tenstorrent have endorsed the standard.

Prof. Kim Jung‑ho of KAIST, dubbed the “father of HBM,” predicts commercial HBF deployment by late 2027‑2028, initially serving AI inference, with HBM + HBF becoming the default AI server configuration. Market forecasts suggest HBM will reach $246 billion by 2030, and HBF could capture a comparable share by 2038.

Remaining Challenges

Intrinsic NAND limitations—finite write endurance and higher access latency—confine HBF to read‑intensive scenarios. Additional hurdles include building a mature software ecosystem, adapting existing stacks, improving TSV yield (current HBM yield ≈ 65 %), and managing thermal and signal‑integrity issues in 16‑layer or deeper stacks.