How to Choose the Right Embedding Model for RAG: A Practical Comparison

Choosing an embedding model is often the first obstacle when building a Retrieval‑Augmented Generation (RAG) system. The author draws on three years of RAG development experience to compare the most common commercial and open‑source models, focusing on concrete metrics such as vector dimension, context length, MTEB benchmark score, price per million tokens, and language coverage.

Core Parameter Comparison

text-embedding-3-large (OpenAI) : 3072‑dim, 8K context, MTEB 64.6%, $0.13/1M tokens, multilingual.

text-embedding-3-small (OpenAI) : 1536‑dim, 8K context, MTEB 62.3%, $0.02/1M tokens, multilingual.

Cohere embed‑v4 : 1024‑dim, 128K context, MTEB 65.2%, $0.10/1M tokens, 100+ languages.

Google text‑embedding‑gecko : 768‑dim, 2K context, MTEB 65.0%, $0.025/1M tokens, multilingual.

BGE‑M3 (BAAI) : 1024‑dim, 8K context, MTEB 63.0%, free (open‑source), 100+ languages.

E5‑mistral‑7b‑instruct : 4096‑dim, 32K context, MTEB 66.6%, free (open‑source), 94 languages.

Voyage‑3 : 1024‑dim, 32K context, MTEB 67.5%, $0.06/1M tokens, multilingual, also offers a code‑specific model (voyage‑code‑3).

Jina‑embeddings‑v3 : 1024‑dim, 8K context, MTEB ~63.0%, $0.02/1M tokens, 94 languages, CC BY‑NC 4.0 license.

Detailed Model Evaluations

OpenAI text‑embedding‑3‑large

Uses Matryoshka Representation Learning to allow truncation directly in the API, which saves a re‑embedding step when a downstream vector store only supports 1024‑dim vectors. The high dimension improves semantic precision, but the price ($0.13 per million tokens) is 6.5× the small version, and storage costs rise sharply. Multilingual performance lags (MIRACL 54.9%). Best for teams that need top‑tier accuracy and already rely on the OpenAI ecosystem.

OpenAI text‑embedding‑3‑small

Dubbed the “cost‑performance king” at $0.02 per million tokens, it raises MTEB from 61.0% to 62.3% over the older ada‑002. It works well for many projects, but multilingual recall on MIRACL drops to 44%, and domain‑specific retrieval can be weaker than Cohere.

Cohere embed‑v4

Offers the highest commercial MTEB score (65.2%) and a massive 128K context window, which reduces the need for aggressive chunking of long documents. Supports 100+ languages and distinguishes query vs. document via the input_type parameter, improving relevance. Pricing ($0.10) is lower than OpenAI’s large model, and vector compression can cut size by 4× with <2% accuracy loss. Drawbacks include a higher price than the small OpenAI model and limited production case studies due to its recent release.

Google text‑embedding‑gecko

At $0.025 per million tokens it is the cheapest commercial option. With 768‑dim vectors and a 2K context window, its MTEB score (~65%) is comparable to Cohere, but documentation and pricing can be inconsistent, as reported by community posts.

BGE‑M3 (BAAI)

Open‑source under MIT license, it outputs dense, sparse, and multi‑vector representations simultaneously, enabling hybrid retrieval strategies. On the MKQA benchmark it achieves Recall@100 = 75.5%, surpassing OpenAI’s latest model. Requires self‑deployment on GPUs, and the CC BY‑NC 4.0 license restricts commercial use.

E5‑mistral‑7b‑instruct

Microsoft’s 7B open‑source model delivers the strongest open‑source MTEB score (66.6%) and a 32K context window, suitable for very long documents. However, it demands high‑end GPUs (V100/A100) and incurs significant deployment and operational costs.

Voyage‑3

Achieves the highest public MTEB score (67.5%) with a flexible dimension choice (256‑2048) and a 32K context window. A specialized code‑retrieval variant (voyage‑code‑3) excels on programming tasks. Pricing ($0.06) is lower than OpenAI’s large model, but the brand is less known.

Jina‑embeddings‑v3

Provides 570M parameters, 94‑language support, and an 8K context window for $0.02 per million tokens. Performance is average (MTEB ~63%). The CC BY‑NC 4.0 license limits commercial deployment.

Selection Recommendations

Start with the cheapest model that meets functional requirements; upgrade only after the workflow is stable. For many cases, text‑embedding‑3‑small is sufficient.

For multilingual workloads, prioritize Cohere embed‑v4 or BGE‑M3; OpenAI’s multilingual claims often underperform in practice.

When data privacy is critical, self‑host BGE‑M3, but factor in GPU electricity and ops costs, which can exceed commercial fees.

For code search, use voyage‑code‑3; generic embeddings perform poorly on programming queries.

Long‑document processing (≥32K context) is best served by Voyage‑3 or E5‑mistral‑7b‑instruct.

Cost Estimation Example

Assuming 1 million documents with 500 tokens each:

text‑embedding‑3‑small: embedding cost ≈ $10/month, storage ≈ $50/month.

text‑embedding‑3‑large: embedding cost ≈ $65/month, storage ≈ $150/month.

Cohere embed‑v4: embedding cost ≈ $50/month, storage ≈ $50/month.

BGE‑M3: embedding cost $0, storage ≈ $50/month, plus GPU electricity ≈ $100/month.

Final Thoughts

There is no one‑size‑fits‑all answer. OpenAI offers the most complete ecosystem and upgrade path; Cohere shines in multilingual and long‑context scenarios; BGE‑M3 provides a free, privacy‑preserving option; Voyage‑3 delivers top performance but with lower brand recognition. The recommended approach is to prototype with a low‑cost model, identify bottlenecks, and then selectively upgrade based on cost, performance, multilingual needs, or data‑privacy priorities.