Spring AI Day 11: Advanced Modular RAG – Query Rewriting, Compression, and Empty‑Context Fallback

1. Limitations of the "naïve" RAG

The original QuestionAnswerAdvisor passes the user’s raw question directly to the retriever, which quickly fails in real scenarios due to three issues:

Ambiguous reference : "What are its parameters?" – the model cannot determine what "its" refers to.

Colloquial phrasing : User language often does not match document wording, leading to poor recall.

No results, no answer : When the retriever returns nothing, the model simply refuses to answer.

To address these problems, the retrieval pipeline must be broken into interchangeable modules – the essence of modular RAG.

2. RetrievalAugmentationAdvisor: A Decomposable RAG

Spring AI provides the spring-ai-rag module. Adding the dependency enables the RetrievalAugmentationAdvisor, which assembles the RAG pipeline from pluggable components. The minimal configuration mirrors the naïve version:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-rag</artifactId>
</dependency>

Advisor ragAdvisor = RetrievalAugmentationAdvisor.builder()
        .documentRetriever(VectorStoreDocumentRetriever.builder()
                .similarityThreshold(0.5)
                .vectorStore(vectorStore)
                .build())
        .build();

String answer = chatClient.prompt()
        .advisors(ragAdvisor)
        .user(question)
        .call()
        .content();

The key difference is that each stage can be swapped out for a custom implementation.

3. Query Rewriting with RewriteQueryTransformer

Before retrieval, the RewriteQueryTransformer uses a large model to turn a spoken question into a retrieval‑friendly form:

Advisor ragAdvisor = RetrievalAugmentationAdvisor.builder()
        .queryTransformers(RewriteQueryTransformer.builder()
                .chatClientBuilder(chatClientBuilder)
                .build())
        .documentRetriever(VectorStoreDocumentRetriever.builder()
                .vectorStore(vectorStore)
                .build())
        .build();

Example: the user asks "I’m learning machine learning, what is LLM?" – the transformer rewrites it to "What is a large language model?", removing noise and immediately improving hit rate.

4. Multi‑turn Compression and Empty‑Context Fallback

Multi‑turn compression uses CompressionQueryTransformer to collapse conversation history into a single independent query:

Query query = Query.builder()
        .text("那它的第二大城市呢？")
        .history(new UserMessage("丹麦的首都是哪？"),
                 new AssistantMessage("丹麦的首都是哥本哈根。"))
        .build();
// Compress to: "丹麦的第二大城市是哪？"
Query standalone = CompressionQueryTransformer.builder()
        .chatClientBuilder(chatClientBuilder).build()
        .transform(query);

Empty‑context fallback adds ContextualQueryAugmenter with allowEmptyContext(true) so that, when no relevant documents are retrieved, the model still generates an answer instead of refusing:

.queryAugmenter(ContextualQueryAugmenter.builder()
        .allowEmptyContext(true)
        .build())

5. Modular RAG Overview

QueryTransformer – rewrites, compresses, or translates the query before retrieval.

DocumentRetriever – fetches documents from a vector store; supports configurable similarity threshold and top‑K.

QueryAugmenter – inserts retrieved documents into the prompt and handles empty‑context strategies.

DocumentJoiner – merges results from multiple retrieval paths.

Key Insight : The naïve RAG is useful for quick prototyping, but production‑grade QA requires modular RAG so each retrieval stage can be tuned individually – query rewriting boosts recall, thresholding reduces noise, and empty‑context fallback prevents refusals.

6. Day 11 Summary

RetrievalAugmentationAdvisor – a decomposable modular RAG component.

RewriteQueryTransformer – rewrites colloquial questions to improve retrieval.

CompressionQueryTransformer – compresses multi‑turn dialogue into a single query.

VectorStoreDocumentRetriever – configurable document retrieval with similarity threshold.

ContextualQueryAugmenter – provides an empty‑context fallback to avoid model refusals.

7. Next Preview

RAG enables AI to "know more" but cannot yet "act" – real‑time weather lookup, ordering, or database operations remain missing. Day 12 will explore Tool Calling, allowing large models to invoke Java methods.