Booking.com's Vector DB Selection: 100M Vectors, 3 Workloads, Why Weaviate Won

From Experiments to Infrastructure

Over several years, embeddings and vector search grew from a few experimental ML use cases at Booking.com into shared infrastructure powering similarity search, semantic filtering, and RAG. The choice of vector database became an infrastructure-level decision requiring predictability and scalability.

Starting Point: OpenSearch with Vector Plugin

Initially, Booking.com chose OpenSearch with its vector plugin for pragmatic reasons: the company already used OpenSearch, AWS offered a managed service, and Terraform could provision it directly. No new technology or operational burden was introduced, allowing the first embedding use cases to ship quickly. This sufficed for early validation but fell short as use cases and scale grew.

When the Pain Started

Multiple pressures converged: the dataset swelled to 100M embeddings; queries evolved from pure KNN to vector search combined with metadata filters; concurrency increased; and user-facing latency requirements tightened. OpenSearch began to struggle — maintaining acceptable performance required constant tuning, clusters grew larger, and both operational burden and cost rose significantly. The team spent more time managing AWS OpenSearch infrastructure than developing AI applications.

Why Public Benchmarks Were Not Trusted

During the search for alternatives, the team reviewed many public benchmarks and comparison articles. While initially helpful, closer inspection revealed they used small datasets, synthetic workloads, and low concurrency — far from Booking.com's production reality of massive embedding datasets, filtered vector search, and high concurrency. At this scale, even small differences in latency or resource efficiency (CPU, memory, cluster size needed to hit target recall and tail latency) have a multiplier effect on user experience and cost. Consequently, the team decided to design their own evaluation that replicated production conditions as closely as possible, aiming not for the best benchmark numbers but for the most predictable behavior under real conditions.

Evaluation Design

Dataset

Built 100M embeddings from the Amazon Reviews 2023 dataset, covering millions of reviews across categories such as Electronics, Baby Products, and Home & Kitchen. The all-MiniLM-L6-v2 model was used for vectorization because it was already in Booking.com's internal model catalog and represents the models actually used in production, offering a good balance of embedding quality and inference speed. Each record also carried metadata like ratings and custom tags to test filtered KNN search performance.

Three Workloads

Pure KNN search: Approximate K-nearest neighbor search over the full 100M dataset. Both systems tuned to ~0.99 recall so latency and throughput differences reflect system efficiency, not retrieval quality.

KNN with filters: Vector similarity search combined with metadata filters (rating ranges, categories, high-cardinality tags). This mirrors real-world usage: filtering logic first narrows the candidate set via metadata, then ANN runs and ranks vectors in the reduced space.

Mixed read/write: Continuous injection of ~20% write operations alongside search requests to simulate real-time updates. Closed-loop mode: each client sends the next request immediately upon receiving a response. This reveals how query latency changes under concurrent write pressure.

Stress Dimensions

Concurrency scaled from 5 threads to 50 to 100. Higher concurrency naturally pushes throughput toward saturation. All tests ran on the full 100M vector dataset to avoid small-dataset bias. Each scenario recorded per-request latency, total throughput, resource utilization, and stability, with special attention to P99 tail latency as the most direct proxy for user experience.

Recall Validation

Because ANN is inherently approximate, the team verified that performance differences were not due to returning worse results. They randomly sampled ~2M vectors from the index as queries and checked whether the query vector itself appeared in the expected top-k nearest neighbors. Both systems were tuned to ~99% recall, ensuring the comparison measured system efficiency alone.

The Gap Is in Storage, Not Indexing

Test results made one thing clear, but the underlying reason deserves deeper explanation. Connor Shorten (Weaviate's Developer Relations lead) highlighted an often-overlooked architectural difference when referencing this test.

Weaviate: Index and storage engine are co-designed. The HNSW graph resides in memory and is persisted via an append-only log; objects and filters are stored in a custom LSM storage. During filtering, bitmaps are generated from the LSM to directly guide HNSW's search path (called ACORN). Writes are cheap appends. The HFresh mechanism goes further by placing vector clusters directly in LSM storage, allowing the index to self-adjust in the background without full rebuilds.

OpenSearch: Built on a segment-based search engine. Each segment builds its own graph independently; queries scan all segments and merges trigger graph rebuilds. The vector index is an add-on layer; the underlying storage engine does not actively cooperate with it. Writes create new segments, queries span multiple segments, and merges cause rebuilds — under load, the latency curve spikes.

This aligns perfectly with Booking.com's observations: OpenSearch performs acceptably at low concurrency, but latency degrades sharply as concurrency and write pressure increase. Weaviate remains nearly flat.

Data: P99 Latency Comparison (Lower Is Better)

Key data points:

Pure KNN: At 5 threads, OpenSearch 84ms vs Weaviate 42ms. At 100 threads, OpenSearch spikes to 1559ms while Weaviate stays at 70ms.

Read + filter: At 100 threads, OpenSearch 241ms vs Weaviate 65ms.

Read + write (20% writes): At 100 threads, OpenSearch 1424ms vs Weaviate 65ms.

The gap is non-linear: higher concurrency accelerates OpenSearch's degradation, while Weaviate remains virtually unaffected.

Not Just Fast — Must Survive Operations

Performance was the primary driver, but a database must also be operable. Booking.com compared deployment flexibility, horizontal scaling, multi-tenancy, backup/restore, monitoring integration, and security compliance. In the feature comparison matrix, Weaviate covered most of Booking.com's operational needs, whereas OpenSearch showed clear gaps in vector-search-specific capabilities such as native multi-vector support and fine-grained filtered search.

Cost

At comparable recall and SLO targets, Weaviate's compute/memory footprint was approximately 40% lower than the OpenSearch baseline. This number directly reflects the efficiency difference — not just fewer machines, but also less tuning and operational time.

Migration: Swap the Backend, Not the Architecture

A decision made early simplified migration enormously. Booking.com's Embedding Service had long abstracted database access behind an internal service; no client touched the storage layer directly. This meant multiple databases could run simultaneously, with the backend switched transparently behind the abstraction. For most teams, migration amounted to a configuration change — no code modifications required.

The Biggest Lesson

Looking back, the team believes the most important takeaway is not about a specific vendor or feature, but about owning the evaluation process. At large scale, workloads are highly idiosyncratic; generic benchmarks rarely reflect real behavior. Running production-like tests yourself is the only way to have confidence in the final choice.

Vector search is now foundational infrastructure for modern ML and AI systems. Treat it like any critical data store: run real tests, abstract the storage layer, optimize for stability over peak numbers. Booking.com did this, saving 40% in cost and avoiding countless future late-night on-call incidents.

Original: How we selected the next vector database at Booking.com (https://booking.ai/how-we-selected-the-next-vector-database-at-booking-com-1e738a5e3bb0)