Vector Database Tutorial: Embeddings, ANN Algorithms & RAG Hands-On Guide

What Is a Vector?

Forget textbook definitions. In this context, a vector is simply a list of numbers representing an object's features. For example, a coffee can be quantified by bitterness (0.8), acidity (0.3), sweetness (0.1), and strength (0.9), yielding the vector [0.8, 0.3, 0.1, 0.9]. Another coffee [0.7, 0.4, 0.2, 0.8] has similar values, so the vectors are close. A milk tea [0.1, 0.1, 0.8, 0.5] differs greatly. Vector retrieval works by converting items to vectors and computing distances to judge similarity.

How Text Becomes Vectors

Embedding models transform text into high-dimensional vectors. For instance, "今天天气真好" becomes a 1536-dimension float array like [0.021, -0.037, 0.089, ..., 0.015]. Semantically similar texts ("天气不错啊") produce nearby vectors; unrelated texts ("数据库怎么优化") produce distant vectors. This enables semantic search: vector retrieval understands that "降本增效" and "成本优化" convey the same meaning, unlike keyword search.

Distance Metrics Between Vectors

Cosine Similarity

Measures whether two vectors point in the same direction, ignoring magnitude. Most common; sufficient for most scenarios.

Euclidean Distance

Straight-line distance in high-dimensional space; smaller distance means more similar.

Dot Product

Considers both direction and magnitude. Used when vector length carries meaning (e.g., popularity weights in recommender systems).

When creating an index, choosing cosine similarity is rarely wrong.

Why a Dedicated Vector Database?

With 100,000 vectors, brute-force comparison against a query vector is feasible but scales poorly. At 1 million vectors it becomes slow; at 100 million it is unusable. Vector databases use Approximate Nearest Neighbor (ANN) algorithms to find similar vectors without exhaustive comparison.

HNSW (Hierarchical Navigable Small World)

Organizes vectors into a multi-layer graph resembling a skip list: coarse search at the top layer, then refinement downward. Fast query speed, high accuracy; the current mainstream default in most vector databases.

IVF (Inverted File Index)

Clusters vectors into groups (centroids). Query first identifies the nearest clusters, then searches only within them. Faster but slightly lower accuracy; suited for very large datasets.

PQ (Product Quantization)

Compresses high-dimensional vectors into low-dimensional codes, trading a small accuracy loss for memory and speed gains. Often combined with IVF.

These algorithms return approximate nearest neighbors with 95%+ recall, delivering orders-of-magnitude speedups.

Getting Started: Hands-On Steps

Step 1: Run a Minimal Demo

Use Python + ChromaDB, which requires no separate service deployment ( pip install chromadb). The following demo creates a collection, adds documents (Chroma automatically embeds them via its default embedding function), and performs a semantic query:

import chromadb
from chromadb.utils import embedding_functions

# 1. Create client and collection
client = chromadb.Client()
ef = embedding_functions.DefaultEmbeddingFunction()
collection = client.create_collection("my_docs", embedding_function=ef)

# 2. Add documents (Chroma auto-embeds text to vectors)
collection.add(
    documents=[
        "Python 是一种解释型的高级编程语言",
        "Java 是一种面向对象的编程语言",
        "今天的午饭是红烧肉盖饭",
        "机器学习是人工智能的一个子领域"
    ],
    ids=["doc1", "doc2", "doc3", "doc4"]
)

# 3. Query: semantic search
results = collection.query(
    query_texts=["有什么好用的编程语言"],
    n_results=2
)
print(results["documents"])
# Output likely returns the Python and Java docs, not the lunch item

Running this demonstrates that vector retrieval finds semantically relevant content even when query and document wording differ completely.

Step 2: Understand the RAG Pipeline

Vector databases' primary use case is Retrieval-Augmented Generation (RAG). Adding an LLM call to the demo yields a minimal RAG system:

# Pseudocode showing core RAG flow
def rag_answer(question):
    # 1. Retrieve: find most relevant docs from vector store
    results = collection.query(query_texts=[question], n_results=3)
    context = "
".join(results["documents"][0])
    # 2. Generate: feed context + question to LLM
    prompt = f"根据以下参考资料回答问题。

参考资料：
{context}

问题：{question}"
    answer = call_llm(prompt)
    return answer

The full RAG flow: user question → embed → vector search → retrieve top-k documents → combine with question as LLM prompt → LLM generates grounded answer. This pattern underpins most enterprise knowledge bases, intelligent customer service, and AI Q&A systems.

Step 3: Switch to a Production-Grade Vector Database

ChromaDB suits learning and small experiments. For production, consider:

Milvus : Open-source, feature-rich, active community, extensive Chinese documentation; good for self-hosted deployments.

Qdrant : Written in Rust, high performance, friendly API, rich filtering support.

Pinecone : Fully managed cloud service; no infrastructure ops required.

Weaviate : Supports multiple quantization methods, built-in embedding model integrations.

The author started with Milvus due to abundant Chinese resources and community solutions.

Step 4: Advanced Topics to Explore After Hands-On Experience

Document chunking strategies : Fixed-length, paragraph-based, or semantic chunking — choice significantly impacts results.

Embedding model selection : OpenAI text-embedding-3 series, open-source BGE series, Sentence-Transformers — each has trade-offs.

Hybrid search : Combining vector search with keyword search (BM25) often outperforms pure vector search.

Index tuning : HNSW parameters ( ef_construction, M), IVF parameters ( nlist, nprobe) — practical tuning comes from real workloads.

These topics become meaningful only after building actual projects.

Common Beginner Pitfalls

Don't obsess over which vector database to pick initially! Core concepts are universal; choose based on data volume, deployment model, and team stack when production needs arise.

Don't neglect embedding model choice! Retrieval quality ceiling is set by the embedding model. A good vs. poor model can differ by 20+ percentage points in recall.

Don't assume vector search solves all search problems! Exact matches (error codes), numeric filtering (time ranges), and sorting (recency) are handled better by traditional databases. Production systems typically blend vector and traditional search.

Don't skip hands-on to read papers first! HNSW, IVF-PQ papers are valuable after you have concrete performance/accuracy intuition from real projects; otherwise they are easily forgotten.

Closing Thoughts

Vector databases boil down to three steps: turn data into vectors, store vectors, quickly find similar vectors. Run the demo first, then backfill theory when problems appear — this order is far more efficient. The demo code above runs in ten minutes; try it now instead of bookmarking and forgetting.