Day 4 of LangChain 7‑Day Series: Build a Full‑Chain RAG QA Bot Step‑by‑Step

Retrieval‑Augmented Generation (RAG) enables a language model to answer queries using external knowledge that was not part of its training data. The workflow consists of three stages: Indexing, Retrieval, and Generation.

1. RAG Three‑Stage Architecture

┌────────────────────────────────────────────────────────────────┐
│                RAG Full Process                │
│                                                    │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│  │  Indexing   │ │ Retrieval │ │ Generation │ │
│  │ (Index)     │ │ (Retrieve)│ │ (Generate) │ │
│  └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│         │            │            │            │
│         ▼            ▼            ▼            │
│ Loader → Splitter  Query → Embedding  Retrieved Docs │
│ → Embedding → VectorDB → Prompt → LLM          │
│ → VectorStore → similarity_search → Answer    │
└──────────────────────────────────────────────────────┘

Stage inputs and outputs:

Indexing : input = raw documents; output = vector database.

Retrieval : input = user query; output = top‑K relevant document chunks.

Generation : input = question + retrieved chunks; output = AI answer.

2. Indexing – Storing Documents in a Vector Store

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

# 1. Load document
loader = PyPDFLoader("company-handbook.pdf")
pages = loader.load()

# 2. Split into chunks (size = 500, overlap = 50)
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
texts = splitter.split_documents(pages)

# 3. Embed and persist (Chroma auto‑persists)
vectorstore = Chroma.from_documents(
    documents=texts,
    embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
    persist_directory="./handbook_db",
)
print(f"Indexing completed: {len(texts)} document chunks")

Practical case : indexing a 200‑page employee handbook produced a ~3 MB vector store with retrieval latency under 100 ms.

3. Retrieval – Finding the Most Relevant Chunks

3.1 Basic similarity_search

# Load existing store
vectorstore = Chroma(persist_directory="./handbook_db",
                    embedding_function=OpenAIEmbeddings(model="text-embedding-3-small"))

# Similarity search (top‑3)
docs = vectorstore.similarity_search(query="How is annual leave calculated?", k=3)
for i, doc in enumerate(docs):
    print(f"Result {i+1}: {doc.page_content[:150]}...")

3.2 Retrieval with Scores

results = vectorstore.similarity_search_with_score(query="What is the resignation process?", k=5)
for doc, score in results:
    print(f"Score: {score:.4f} | Content: {doc.page_content[:100]}")

3.3 Top‑K vs. MMR (Maximum Marginal Relevance)

# Top‑K (k = 3)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
docs = retriever.invoke("How to apply for overtime pay?")

# MMR (k = 3, fetch_k = 10, lambda_mult = 0.5)
retriever = vectorstore.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 3, "fetch_k": 10, "lambda_mult": 0.5},
)
docs = retriever.invoke("What employee benefits does the company offer?")

MMR returns a broader coverage of documents while avoiding near‑duplicate results.

4. Generation – Building RAG Chains with LCEL

4.1 Single‑turn RAG

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

vectorstore = Chroma(persist_directory="./handbook_db",
                    embedding_function=OpenAIEmbeddings(model="text-embedding-3-small"))
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
llm = ChatOpenAI(model="Pro/MiniMaxAI/MiniMax-M2.5", temperature=0)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a professional Q&A assistant. Answer using the reference documents.

Reference docs:
{context}"),
    ("human", "{question}"),
])

rag_chain = (
    {"context": retriever | (lambda docs: "

".join(d.page_content for d in docs)),
     "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)
result = rag_chain.invoke("What is the length of the probation period?")
print(result)

4.2 Multi‑turn Conversation

from operator import itemgetter
from langchain_classic.memory import ConversationBufferMemory

memory = ConversationBufferMemory(return_messages=True)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a professional Q&A assistant. Answer using reference docs.

Reference docs:
{context}

Conversation history:
{history}"),
    ("human", "{question}"),
])

rag_chain = (
    {"context": itemgetter("question") | retriever | (lambda docs: "

".join(d.page_content for d in docs)),
     "history": itemgetter("history"),
     "question": itemgetter("question")}
    | prompt
    | llm
    | StrOutputParser()
)

def chat(question: str) -> str:
    history = memory.load_memory_variables({})["history"]
    history_text = "
".join(f"{m.type}: {m.content}" for m in history) if history else ""
    result = rag_chain.invoke({"question": question, "history": history_text})
    memory.save_context({"input": question}, {"output": result})
    return result

print(chat("How is annual leave calculated?"))
print(chat("After three years, how many days do I have?"))

4.3 LCEL Chain Types Comparison

stuff

(single Prompt): suitable for < 5 chunks; low token usage; fails with larger data. map_reduce: handles large data; requires multiple LLM calls; slower. refine: iterative refinement; higher answer quality; slower and higher token count. mmr (diversity retrieval): avoids duplicate results; broader coverage; slightly more complex to implement.

5. Real‑World Case: Internal Knowledge‑Base Chatbot

import os
from operator import itemgetter
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_classic.memory import ConversationSummaryBufferMemory
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

os.environ["OPENAI_API_KEY"] = "your-api-key"

vectorstore = Chroma(persist_directory="./company_kb",
                    embedding_function=OpenAIEmbeddings(model="text-embedding-3-small"))
llm = ChatOpenAI(model="Pro/MiniMaxAI/MiniMax-M2.5", temperature=0)
memory = ConversationSummaryBufferMemory(llm=llm, memory_key="chat_history", max_token_limit=1000, return_messages=True)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a professional Q&A assistant. Answer using reference docs.

Reference docs:
{context}

Conversation history:
{history}"),
    ("human", "{question}"),
])

rag_chain = (
    {"context": itemgetter("question") | retriever | (lambda docs: "

".join(d.page_content for d in docs)),
     "history": itemgetter("history"),
     "question": itemgetter("question")}
    | prompt
    | llm
    | StrOutputParser()
)

questions = [
    "What is the expense reimbursement process?",
    "Which invoices need to be prepared?",
    "How many days does approval usually take?",
]
for q in questions:
    history = memory.load_memory_variables({})["history"]
    history_text = "
".join(f"{m.type}: {m.content}" for m in history) if history else ""
    result = rag_chain.invoke({"question": q, "history": history_text})
    memory.save_context({"input": q}, {"output": result})
    print(f"Q: {q}
A: {result}
")

Result : a startup that deployed this RAG bot saw internal administrative queries drop by 60 % and employee wait time shrink from four hours to real‑time responses.

6. Day 4 Recap

Indexing : core API = Chroma.from_documents() Retrieval : core API = vectorstore.as_retriever() (optionally with MMR via search_type="mmr")

Generation : LCEL chain pattern = retriever | prompt | llm Optimization : use MMR ( search_type="mmr") for diverse results

Memory : ConversationSummaryBufferMemory combined with LCEL

Related Links

LangChain RAG tutorial: https://python.langchain.com/docs/tutorials/rag/

LCEL RAG guide: https://python.langchain.com/docs/concepts/lcel/

LangChain Memory docs: https://python.langchain.com/docs/concepts/memory/