Interviewer's Guide: Four Key Factors for Choosing an Embedding Model

Interview Scene

Choosing an embedding model requires considering four indispensable factors; picking the wrong one makes any downstream retrieval tuning futile.

① Language Support

Chinese scenarios need models optimized for Chinese, as English‑trained models perform poorly. Recommended open‑source options include:

BGE series (bge-large-zh) : currently the best open‑source Chinese embedding.

M3E : strong multilingual support, ideal for mixed Chinese‑English use.

Text2Vec : lightweight Chinese model suitable for rapid prototyping.

English scenarios can use a broader set of models, such as: text-embedding-ada-002 (1536‑dim, high quality).

Sentence‑Transformers series (e.g., all‑MiniLM‑L6‑v2).

Key principle : use a model optimized for the language you process; a single model rarely covers all languages effectively.

② Vector Dimension

Higher dimensions increase expressive power but also raise storage and computation costs.

384 dim – lightweight, ~1.5 KB per vector, suited for quick validation.

768 dim – balanced choice, most commonly used, ~3 KB per vector.

1024 dim – high‑precision scenarios, ~4 KB per vector.

1536 dim – OpenAI ada‑002, best performance, ~6 KB per vector.

For a corpus of 1 million documents, 1536‑dim vectors consume about 3 GB more storage than 768‑dim vectors, a noticeable difference at large scale.

③ Context Length

The maximum token length a model can handle must cover the size of your document chunks; otherwise, truncation occurs.

Common lengths: 512, 1024, 2048, 8192 tokens.

If your chunks are 512 tokens, a model that supports fewer than 512 tokens will truncate the input.

Long‑document use cases (e.g., full PDF pages) require models with sufficiently long context windows.

④ Performance Metrics

Evaluate models on standard benchmark sets; each metric reflects a different aspect of retrieval quality:

MRR : inverse rank of the first relevant document; higher is better.

NDCG : position‑weighted ranking metric; rewards placing relevant documents near the top.

Recall@K : proportion of relevant documents within the top K results; typical K values are 3, 5, 10.

Practical Selection Advice

Start with an open‑source model to validate effectiveness; only switch to a commercial API if results are unsatisfactory. For Chinese tasks, bge-large-zh (768 dim) offers a good balance between accuracy and cost.

Real‑World Use Cases

Multilingual e‑commerce customer service : Chinese and English queries can be handled by M3E, eliminating the need for separate embedding services.

High‑precision legal retrieval : Choose 1536‑dim ada‑002 or bge‑large to prioritize accuracy despite higher storage cost.

Lightweight personal knowledge base : For small, low‑accuracy needs, a 384‑dim lightweight model suffices with near‑zero deployment cost.

Interviewer's favorite question : How do you evaluate an embedding model? Answer: Use public benchmark suites such as C‑MTEB or MTEB, or construct a test set from business data and compute retrieval accuracy and recall.

Remember: Embedding model selection = language support + vector dimension + context length + performance metrics; consider all four, start with open‑source, then evaluate commercial options.