RAG in Action: Enabling AI to Answer Using Your Own Documents with Spring AI

RAG overview

Retrieval‑Augmented Generation (RAG) consists of three stages:

Retrieval : query a vector store to obtain document fragments relevant to the user question.

Augmentation : splice the retrieved fragments into the prompt as contextual evidence.

Generation : the language model generates an answer conditioned on the augmented prompt.

This workflow mitigates hallucination and knowledge‑staleness by grounding the model’s output in factual snippets.

QuestionAnswerAdvisor usage

Add the Spring AI vector‑store advisor dependency (Maven coordinates escaped for HTML):

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-advisors-vector-store</artifactId>
</dependency>

Instantiate a ChatClient with the advisor and invoke a single‑line prompt:

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore).build())
        .build();

String answer = chatClient.prompt()
        .user("我们的退货政策是几天？")
        .call()
        .content(); // answer is sourced from the policy document stored in the vector store

The client internally vectorises the question, retrieves matching fragments, injects them into the prompt, and forwards the augmented prompt to the model.

Retrieval parameter tuning

Control the number and relevance of returned fragments via SearchRequest:

var qaAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
        .searchRequest(SearchRequest.builder()
                .similarityThreshold(0.8) // keep fragments with relevance > 0.8
                .topK(6) // retrieve up to 6 fragments
                .build())
        .build();

topK

: a small value may miss key information; a large value can introduce noise and increase token usage. similarityThreshold: a high threshold may return no fragments; a low threshold may admit irrelevant content.

Full RAG pipeline

【Offline – Build Knowledge Base】
Document → Read → Chunk → Vectorise → Vector Store

【Online – Answer Questions】
Ask → Vectorise → Retrieve relevant fragments → Append to prompt → LLM → Answer

Combining advisors (memory + RAG)

Multiple advisors can be attached to a single ChatClient so that conversation history is retained while the knowledge base is consulted:

var chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(
                MessageChatMemoryAdvisor.builder(chatMemory).build(), // memory
                QuestionAnswerAdvisor.builder(vectorStore).build())   // RAG
        .build();

Key concepts

RAG = Retrieval + Augmentation + Generation, enabling fact‑based answers. QuestionAnswerAdvisor provides a ready‑made RAG implementation. topK and similarityThreshold are primary knobs for retrieval quality.

Advisors are plug‑and‑play; they can be freely combined (e.g., memory plus RAG).

Related links

RAG (Retrieval‑Augmented Generation): https://docs.spring.io/spring-ai/reference/api/retrieval-augmented-generation.html
Advisors API: https://docs.spring.io/spring-ai/reference/api/advisors.html