Volcano Milvus Hits Nearly 3× VectorDBBench Leader in Retrieval Speed

Background and Challenge

Large language models increase memory and hardware costs, making hardware expense a primary bottleneck for scaling vector retrieval. In massive vector workloads, balancing throughput, latency, recall, and data‑import efficiency under a fixed budget is a core difficulty.

Algorithmic Foundations

Volcano Milvus integrates DiskANN and RaBitQ. DiskANN provides an in‑memory mode for maximum speed and a disk‑based mode for ultra‑large, low‑memory workloads ( https://github.com/microsoft/DiskANN). RaBitQ is a state‑of‑the‑art quantization method; its authors extended it (Extended‑RaBitQ) to multi‑bit representations, improving distance‑estimation accuracy while keeping storage low ( https://arxiv.org/abs/2409.09913). Milvus combines the two so that coarse recall uses aggressive compression and candidate refinement uses higher‑precision quantization.

In‑Memory Layout Optimizations

For high‑performance in‑memory retrieval, Milvus redesigns DiskANN’s graph structures to align with CPU cache lines. RaBitQ vectors, adjacency lists, and candidate metadata are stored contiguously in a cache‑friendly format, allowing the query thread to batch distance calculations with minimal random memory accesses. This layout does not change recall but reduces memory‑fetch time, freeing CPU cycles for distance computation and graph traversal.

Query‑Path Slimming and Runtime Tweaks

Fine‑grained analysis of the search pipeline identified unnecessary memory allocations, intermediate objects, and redundant copies. Trimming these hot‑path elements lowers base request latency, increasing overall throughput under the same resource budget.

Prioritize lightweight RaBitQ vectors during neighbor expansion.

Optimize batch distance estimation to keep data in CPU registers.

Use large pages and pool‑based indexnode allocation to reduce page‑table overhead.

Benchmark Methodology (VectorDBBench)

VectorDBBench evaluates write, search, and filtered‑search workloads. All tests were limited to a fixed monthly cost of $1,000 (≈ 7,105 CNY) and a total of 59 CPU cores (59C) for the online query chain.

Dataset: 10 M vectors from Cohere, dimension 768.

Similarity metric: cosine.

Top‑K: 100.

Recall target: 93.9 %.

Client concurrency: 250.

Hardware: Volcano Engine ECS ecs.g4al.16xlarge (proxy 8C8G, mixcoord 1C2G, querynode 48C48G, datanode 2C4G, indexnode pooled).

Results

QPS = 20,420 (≈ 3× the VectorDBBench top score).

p99 latency = 2.7 ms, average latency = 2.5 ms.

Recall = 93.9 %.

Import time ≈ 4,515 s.

Compared with the open‑source Milvus baseline, the current version delivers more than 5× performance improvement and reaches the VectorDBBench leader with roughly one‑third of the memory, disk, and compute resources.

Conclusion

Cost‑efficiency in the VectorDBBench test results from algorithmic advances (DiskANN + Extended‑RaBitQ) and deep engineering optimizations (in‑memory layout, query‑path slimming, runtime tuning). When scaling to billions of vectors, this efficiency translates into potential monthly savings of tens of thousands of yuan, making Volcano Milvus a compelling choice for high‑throughput, low‑latency, high‑recall vector retrieval in real‑world online services.