How Alibaba Cloud Elasticsearch’s Cloud‑Native Vector Engine Tops VectorDBBench

Alibaba Cloud Elasticsearch on ES 9.4 rebuilds the end‑to‑end vector retrieval path with the FalconSeek HNSW kernel, covering candidate generation, batch computation, and original‑vector re‑ranking. In VectorDBBench, the system reaches 82,520.18 QPS at roughly 0.98 recall, with a P99 latency of 1.8 ms.

Why a new design is needed

As AI applications such as RAG, semantic search, and multimodal retrieval move to production, vector dimensions (e.g., 768‑D general semantic vectors and 1024‑D biomedical text vectors) and data sizes grow from millions to tens of millions. The challenge shifts from merely finding similar vectors to meeting high recall, low latency, high concurrency, continuous writes, filtered queries, and cost constraints. Optimizing a single distance function no longer yields stable end‑to‑end gains.

FalconSeek HNSW: Core techniques

Quantized data for candidate recall, original vectors for re‑ranking : Indexes store both quantized representations (used during HNSW graph traversal to compute approximate distances and maintain the candidate queue) and the full‑precision vectors (read later for exact similarity scoring). The size of the candidate set is configurable to control the workload of the re‑ranking stage.

Batch computation and instruction‑level parallelism : Distance calculations for a batch of neighbor nodes are grouped and executed with SIMD vector instructions, while the engine schedules instruction‑level parallelism and prefetching across different candidates.

Hot‑data layout based on access patterns : Graph adjacency lists, quantized vectors, and auxiliary metadata are stored in a compact read‑only layout that respects the traversal order, keeping frequently accessed data physically close.

On‑demand batch re‑ranking of original vectors : After candidate IDs are produced by the HNSW graph, the engine fetches the corresponding original vectors and computes exact similarities in batch, then sorts and truncates to the final Top‑K.

Integration with Elasticsearch segment lifecycle : FalconSeek reuses Elasticsearch’s shard, segment, refresh, and merge mechanisms. New writes become searchable after refresh, and merges generate new segments that include the graph structure, quantized data, and original vectors, which are atomically published.

The combined effect of these optimizations yields the reported 82 K QPS and 1.8 ms P99 latency, not because of a single tweak but due to the coordinated improvements across candidate generation, batch computation, data layout, and segment management.

Engine enhancements for different scale and resource goals

Flat (exact scan) : Suitable for small datasets where precision and simplicity matter; scans all vectors without an approximate graph.

falconseek_hnsw (performance‑first) : Uses quantized recall followed by original‑vector re‑ranking, ideal when hot data fits in memory and low latency is critical.

bbq_hnsw (memory‑first) : Retains the HNSW graph but compresses the search vectors to 1‑bit, reducing memory footprint at the cost of additional re‑ranking.

bbq_disk (capacity‑first) : Replaces the full graph with a clustering‑based index that operates on disk or object storage, enabling billion‑scale vectors without requiring the entire graph in memory.

Agent‑centric multi‑tenant scenario

The AI Engine edition of Alibaba Cloud Elasticsearch adds a system‑level route for billions of tenants and trillions of vectors, using OSS for persistent storage, stateless compute nodes, and a cache hierarchy of memory and SSD. It preserves Elasticsearch’s full‑text, vector, filter, and aggregation capabilities while supporting per‑tenant isolation.

Full‑stack AI search performance

Beyond vector retrieval, the platform optimizes traditional search, full‑text, and multimodal pipelines. Benchmarks show:

Rally Big5: Query latency drops from 48.609 ms to 7.076 ms (≈6.9×) after enabling FalconSeek.

Tantivy Search Benchmark: Average query time of 912 µs, 2.00× faster than Lucene 10.4.0.

MMEB VisDoc: Ops‑Colqwen3‑4B achieves a Visdoc‑Overall score of 84.12, ranking first.

These results demonstrate that production‑grade AI search must coordinate vector recall with full‑text, filter, and multimodal signals within a unified query framework.

Conclusion

The performance ceiling of vector search is determined by the entire retrieval pipeline: candidate generation, hot‑data layout, original‑vector access and re‑ranking, and seamless integration with segment lifecycle. FalconSeek HNSW’s success in VectorDBBench proves that a well‑engineered end‑to‑end path can simultaneously boost throughput, maintain high recall, and keep tail latency low, while fitting naturally into Elasticsearch’s existing data and query ecosystem.