Understanding Vector Databases and Embedding Techniques

Recently a friend was asked about vector databases in an interview and didn't know what they were.

With the rise of ChatGPT and AI products, vector databases have gained attention, though they have existed for a long time and are used by many companies; recent AI hype has brought companies like Pinecone into the spotlight.

What are Vectors and Vectorization

Vector databases store vectors as their primary data.

A vector is a numeric representation of an object, e.g., a 2‑D vector (x, y) or a 3‑D vector (x, y, z), and can have many dimensions.

The main use case is similarity search, which matches a query to the most similar results, similar to fuzzy search but based on embedding + index rather than tokenization + index.

Indexes are essential for storage.

Embedding

OpenAI provides an embedding model called Ada that converts data into low‑dimensional dense vectors.

Embedding represents an object with a dense vector so that distances reflect similarity.

An embedding is a multi‑dimensional array of numbers that can be generated from text, audio, video, etc., and stored in a vector database.

For example, the vector for man might be [0.1,0.2,0.1] and for woman [0.3,0.1,0.1].

These vectors occupy positions in a vector space, showing relationships such as man‑woman, king‑queen, China‑Beijing.

After vectorizing various content, similar items cluster together, e.g., animal‑related queries will not match athlete data.

The embedding process relies on large pretrained models and neural networks, often accessed via paid services like OpenAI’s Ada.

Embedding must handle multilingual matching, e.g., the English word apple should match the Chinese 苹果 , and even handle sentiment variations.

Current mainstream embedding methods fall into three categories:

Matrix Factorization

Matrix factorization maps a high‑dimensional matrix into the product of two low‑dimensional matrices, alleviating sparsity.

NLP‑based Methods

Natural Language Processing techniques convert words or phrases into low‑dimensional vectors, placing semantically similar terms close together. Common methods include:

Word2vec

GloVe (Global Vectors for Word Representation)

FastText

Graph‑based Methods

For data with graph structures such as social networks or knowledge graphs, graph embedding maps nodes to low‑dimensional vectors. Common algorithms include:

DeepWalk

Node2vec

Metapath2vec

Characteristics of Vector Databases

Vector databases must handle massive amounts of data, unlike traditional relational databases.

High availability and scalability architecture.

Compute‑intensive workloads requiring powerful hardware acceleration.

High concurrency and low latency.

Application Scenarios

The core function is similarity matching, leading to use cases such as:

Semantic text search.

Image, audio, and video search (e.g., image‑by‑image search, voice fingerprinting, song identification).

Recommendation systems that suggest items with highest similarity to user profiles.

Anomaly detection, such as face recognition where low similarity indicates a non‑match.

While the AI boom introduces many new techniques, most developers rely on existing APIs rather than building embeddings from scratch.

Nevertheless, staying informed is worthwhile because vector databases may become relevant to future products.