HELMSMAN: Redefining Large-Scale Vector Retrieval on All‑Flash Servers (OSDI 2026)

Search, recommendation, and advertising services at scale require billions of high‑dimensional vectors to be queried within a 5‑10 ms latency budget, traditionally relying on in‑DRAM graph indexes such as HNSW. As vector data grows near‑doubling each year, DRAM consumption reaches petabyte levels, incurring multi‑million‑dollar annual hardware costs and making pure‑memory deployments unsustainable.

Modern NVMe SSDs offer bandwidth comparable to DRAM at a fraction of the cost, but naïvely moving graph indexes to SSD fails because graph traversal depends on serial I/O: each neighbor lookup requires the result of the previous SSD read, preventing full bandwidth utilization and inflating tail latency.

HELMSMAN proposes a clustering‑based ANN design that first locates a set of centroid candidates in memory, then issues batch reads of the corresponding cluster lists from SSD, finally computes distances locally. This batch‑oriented access pattern fits SSD bandwidth and eliminates the serial dependency of graph searches.

To exploit SSDs efficiently, HELMSMAN builds a custom ANNS storage stack on top of SPDK, bypassing the Linux kernel, file system, and block layer. Fixed‑size clusters are stored in raw NVMe logical blocks and allocated via a unified chunk allocator, allowing a single I/O submission to retrieve an entire cluster list.

The system introduces Leveling‑Learned Search Pruning (LLSP). A router model predicts the appropriate search level based on query features and top‑k, and a GBDT model decides the exact nprobe before any SSD read, ensuring that pruning happens prior to I/O and preserving batch reads. This adaptive strategy reduces unnecessary reads for easy queries while allocating enough probes for hard queries.

Construction is split into three stages: (1) GPU‑accelerated coarse k‑means generates initial centroids; (2) an elastic CPU pool performs cluster splitting, load balancing, and boundary padding; (3) multi‑core CPUs merge shards, build the centroid graph, train the LLSP model, and materialize the final index. The pipeline runs on RED‑Ray for task scheduling and Daft for parallel data processing, leveraging idle CPU capacity via Virtual Kubelet.

Experiments on public benchmarks and Red’s production workloads show that HELMSMAN delivers 2‑16× higher throughput than DiskANN, Starling, PipeANN, and SPANN, reaching up to 85% of pure‑memory HNSW throughput while maintaining 5‑10 ms average latency and strict tail‑latency guarantees. SSD bandwidth utilization climbs to ~85% on Gen4 and ~70% on Gen5 devices. In production, ~40 all‑flash servers replace the previous ~35,000 CPU cores and 350 TB DRAM, cutting hardware cost by over 90%. Indexes of 0.1 B vectors are built in under one hour, and 10 B‑scale indexes are rebuilt in 4‑7 hours, a tenfold speedup.

Overall, HELMSMAN demonstrates that when vector scales reach tens of billions and DRAM costs become prohibitive, a carefully engineered SSD‑first ANN pipeline—combining clustering, user‑mode storage, learned pruning, and heterogeneous construction—can meet strict online SLAs while dramatically reducing cost, establishing a viable path for future large‑scale vector retrieval systems.