Everything You Need to Know About Retrieval‑Augmented Generation (RAG)

A fictional programmer struggles to get useful answers from a large language model when he feeds an entire year‑long document about a boss; the model returns generic replies. Realising that only the relevant fragment is needed, he discovers Retrieval‑Augmented Generation (RAG), which combines information retrieval with text generation.

RAG works in three stages: Retrieval – the system searches an external knowledge base (documents, manuals, news, etc.) for pieces related to the user query; Augmentation – the retrieved snippets are appended to the original prompt; Generation – the large language model produces a response grounded in the augmented context.

To decide which fragments are relevant, the article proposes embedding the texts into high‑dimensional vectors. It illustrates this with four example sentences (A‑D) about the programmer’s clothing style, showing that the distance between vectors for A and B is smallest, indicating higher relevance. Embedding models such as OpenAI’s text-embedding-3-small (1536‑dimensional) and text-embedding-3-large (3072‑dimensional) are mentioned.

When documents are too large, the workflow applies Chunking (splitting by characters, paragraphs, or sentences), then computes an embedding for each chunk, turning them into fixed‑length vectors (often called "vectors" or "embeddings"). These vectors are stored in a vector database; a query is also embedded, and the database returns the nearest n chunks, which are then sent to the language model.

Following these steps—chunking, vectorizing, storing in a vector DB, retrieving the most similar chunks, and augmenting the prompt—completes a functional RAG pipeline.