Choosing a Vector Database: Pinecone for Production, Chroma for Prototyping, Weaviate for Hybrid Search

What Vector Databases Actually Do

Vector databases store numeric embeddings (e.g., 768‑ or 1,536‑dimensional arrays) generated from text, images, or audio, and build indexes that enable fast nearest‑neighbor searches. In Retrieval‑Augmented Generation (RAG), a user query is first embedded, then the database returns the most semantically similar text fragments, which are fed to a large language model.

Chroma: Starting from a Prototype

Chroma is an open‑source solution that can be installed with pip install chromadb. It runs either in‑memory or persists to disk, allowing a usable vector store to be built within minutes.

import chromadb
from chromadb.utils import embedding_functions

client = chromadb.PersistentClient(path='./my_db')
ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key='your-key', model_name='text-embedding-3-small')
collection = client.get_or_create_collection('docs', embedding_function=ef)

# Add documents
collection.add(documents=['doc1 text', 'doc2 text'], ids=['id1','id2'])

# Query
results = collection.query(query_texts=['your question'], n_results=5)

Chroma is not cloud‑native; scaling beyond a single machine or beyond roughly one million documents requires manual server management and migration. The API is clean, but the migration effort must be planned.

Pinecone: The Production‑Ready Choice

Pinecone provides a fully managed cloud service—no servers, memory, or replica management are needed. The free tier supports about one million 1,536‑dimensional vectors, sufficient for many small applications; paid tiers scale to billions of vectors.

from pinecone import Pinecone

pc = Pinecone(api_key='your-pinecone-api-key')
index = pc.Index('my-index')

# Upsert (embedding must be prepared separately)
index.upsert(vectors=[('id1', embedding_vector, {'text': 'doc text'})])

# Query
results = index.query(vector=query_embedding, top_k=5, include_metadata=True)

The free tier is useful, but costs grow with vector count and query volume. For a startup handling ~10,000 queries per day, expenses remain manageable; large‑scale deployments can become costly, so it is advisable to abstract the retrieval logic behind a clear interface to allow future migration.

Weaviate: Hybrid Search for Mixed Needs

Pure semantic search can miss exact keyword matches (e.g., searching for "RFC 7519"), while pure keyword search ignores semantic similarity. Weaviate combines cosine similarity with BM25 keyword matching, allowing weighted hybrid queries.

import weaviate

client = weaviate.connect_to_wcs(cluster_url='…', auth_credentials=…)
collection = client.collections.get('Document')

# Hybrid query: combine semantic + keyword
results = collection.query.hybrid(
    query='your question',
    alpha=0.5,  # 0 = keywords only, 1 = semantics only, 0.5 = balanced
    limit=5
)

Hybrid search shines when the knowledge base contains technical documentation, API references, or content with specific identifiers, code, or model numbers; for generic text the benefit is modest and the added complexity may not be justified.

Frequently Asked Questions

Which vector database should I use for my first project?

Chroma is the obvious starter: pip install, local execution, zero configuration, and free. When the project outgrows local limits, migrating to Pinecone or Weaviate typically takes only a few hours, provided the API surface remains consistent.

Do I need a vector database for RAG, or can a regular database suffice?

PostgreSQL with the pgvector extension can perform approximate nearest‑neighbor search and works well for under a million vectors, especially on hosted services like Supabase. Dedicated vector databases show performance advantages at larger scales.

Which embedding model should I choose?

OpenAI and Google APIs offer reliable, low‑cost embeddings (≈ $0.02 per million tokens). For on‑premise, privacy‑focused scenarios, Ollama running nomic-embed-text is a free option. When cost is not a concern and maximum quality is desired, OpenAI's text-embedding-3-large or Cohere's embed‑v3 are recommended.