Understanding Cosine Similarity in Large Language Models

Principle

Cosine similarity measures the cosine of the angle between two vectors A and B: cos(θ) = (A·B) / (||A|| * ||B||). The result ranges from -1 to 1, with values closer to 1 indicating that the vectors point in the same direction and are therefore more similar.

What It Does in LLMs

In large language models, text such as sentences, words, or documents is first converted into high‑dimensional embeddings (e.g., using OpenAI Embedding or Sentence‑BERT). The model then computes cosine similarity between the query embedding and each document embedding to find the most semantically related pieces of text.

The top‑scoring passages are fed into the generation step, which is the basis of Retrieval‑Augmented Generation (RAG). This process allows the model to produce answers that are more accurate and grounded.

Key Characteristics

Ignores text length : similarity depends only on direction, not on how long the sentences are. Example: “我吃饭了” and “我刚刚吃了一碗米饭” receive a high score.

Considers direction, not magnitude : sentences with the same meaning but different wording score highly. Example: “公司福利有哪些” vs. “公司提供什么待遇”.

Works with high‑dimensional data : vectors of 512 or 768 dimensions can be compared directly.

Fast and sortable : cosine similarity can be computed quickly and used to rank many texts, enabling millisecond‑level search even over billions of vectors when combined with approximate nearest‑neighbor libraries such as FAISS.

Why It Matters

Core metric for AI retrieval systems, knowledge‑base Q&A, and RAG pipelines.

Replaces keyword matching with semantic understanding.

Provides a seamless bridge between retrieved knowledge and LLM generation, reducing hallucinations.

Easy to implement and high‑performance, especially with approximate algorithms.

Intuitive Analogy

Imagine asking someone for directions. The question “Are we heading the same way?” mirrors cosine similarity’s focus on direction rather than distance. In AI, “same direction” means semantically similar content.

Python Demo

from sklearn.metrics.pairwise import cosine_similarity
from sentence_transformers import SentenceTransformer

# 示例：两个句子
texts = ["我喜欢猫", "我讨厌猫"]

# 转换为向量
model = SentenceTransformer('paraphrase-MiniLM-L6-v2')
embeddings = model.encode(texts)

# 计算余弦相似度
similarity = cosine_similarity([embeddings[0]], [embeddings[1]])
print(f"相似度：{similarity[0][0]}")

In this example, two Chinese sentences are encoded into embeddings and their cosine similarity is printed, illustrating how the metric quantifies semantic closeness.

Bottom Line

Cosine similarity acts as a “semantic radar” for large language models, helping them decide whether two sentences convey the same meaning. Its simplicity, efficiency, and effectiveness make it an indispensable tool for generative AI.