High Bandwidth Flash (HBF): A 10x Cost Advantage for Cloud AI Inference?

Conference Background

The 2026 Advanced Storage Conference (ASC), organized by the Open Data Center Committee (ODCC), was held in Shanghai on October 21-22. Alibaba Cloud senior technical expert Wu Guanying delivered a keynote titled "Opportunities and Challenges of HBF in Cloud AI Inference — From Media to Near-Storage Compute Hardware-Software Co-Design Exploration," systematically sharing technical judgment, industry tracking, and engineering practice on High Bandwidth Flash (HBF).

AI Inference Drives the Capacity Wall

As large-model inference moves toward long-context and agentic workloads, demand for KV Cache and model-weight storage capacity surges. A single million-token context session can generate hundreds of gigabytes of KV Cache, far exceeding the economic capacity of HBM. This "capacity wall" and cost-performance ratio have become the core contradictions in AI inference infrastructure.

HBF: A New Media for the Capacity Layer

HBF (High Bandwidth Flash) emerges as a new medium in this context. Built on NAND flash with advanced packaging and UCIe interconnect, it achieves TB/s-level bandwidth within a ~100 mm² cube. SanDisk first announced the HBF concept in February 2025. Key first-generation specs:

Single-stack capacity: ~512 GB

Read bandwidth: ~1600 GB/s

Cost comparable to HBM, but $/GB ≈ 1.07 vs. HBM3E ≈ 11.5 — a ~10x unit-cost advantage

Capacity 8-16x HBM

These characteristics make HBF ideal as a "capacity layer" for full model weights, full KV Cache, historical/idle session KV, and prefix cache — data that is read-heavy and write-light.

Industry Progress: From Concept to Standardization and Sampling

Vendor roadmaps:

SanDisk (OCP HBF spec initiator): first die taped out August 2026; HBF-equipped inference devices planned for 2027 sampling.

SK Hynix (co-author of spec): plans HBF cube samples early 2028.

Multiple vendors exploring near-GPU NAND, in-package TSV stacking (up to 32 dies), and custom zNAND-O.

Standardization: OCP formed the HBF Work Group in February 2026, released v0.7 spec, and the August 2026 version defines 16 UCIe channels with three bandwidth tiers (0.38–3.1 TB/s). Volume production is expected in 2028.

Alibaba Cloud's engagement: The team entered in November 2025 and within one year completed concept evaluation, full-system simulation, spec definition, decode-acceleration product design, PoC prototype validation, and software-stack integration. A 2027 sampling R&D plan has been initiated.

Four Key Challenges for Cloud Deployment

Media inherent limitations: Read latency in microseconds, low write bandwidth, limited endurance/reliability. The base die sits at the bottom; heat must traverse the NAND stack, and higher temperature increases refresh traffic, further reducing endurance.

Software stack complexity: Requires data placement and eviction across a four-tier storage hierarchy (HBM → HBF → SSD), KV eviction policies, and three-layer adaptation for inference-framework KV, attention, and routing.

Ecosystem and supply chain: Interface spec not yet frozen; controller IP, drivers, and operator libraries lack ready solutions. High-bandwidth memory devices also face export controls.

Model architecture as a moving target: Today's interface and NAND topology may not be optimal next year, necessitating co-adaptation between models and HBF.

Methodology: Trace-Driven, Constraint-Based Simulation

Using real inference load traces from the Bailian production environment, combined with the HiSim high-fidelity simulator — which integrates a real inference engine, xPU simulation, and HBF simulation — the team evaluates designs under four constraints: performance, endurance, reliability, and power. Core metrics are cost per million tokens ($/M token), five-year TCO, and throughput. Simulation results feed back into spec definition, scenario selection, and architectural trade-offs.

Conclusion and Call for Ecosystem Collaboration

From a CSP perspective, HBF provides a new path that balances large capacity with high bandwidth, significantly lowering AI inference cost. Wu Guanying emphasizes that HBF has become a structural variable in the storage-chip industry, potentially reshaping supply-demand dynamics and competitive positioning. China's "15th Five-Year Plan" explicitly calls for HBF R&D, giving the domestic storage industry a rare window of opportunity.

He stresses that HBF maturation requires full-stack collaboration — "many hands make light work" — and invites storage-media vendors, compute-chip makers, inference-framework developers, and CSPs to work together. The transition from HBM dependence to an HBF capacity layer, and from media innovation to near-storage compute hardware-software co-design, cannot be achieved by any single party. Only through open, co-designed cooperation can HBF's cost advantage be converted into a sustainable AI inference cost advantage. Alibaba Cloud plans to partner with the ecosystem to drive large-scale HBF deployment in cloud AI inference, building a more cost-effective and sustainable storage foundation for the AI era.