Retrieval‑Augmented Generation (RAG) for Office Applications: Architecture, Challenges, and Practical Practices

With the rapid development of large language models, the gap between Chinese models and OpenAI’s offerings is narrowing, yet practical deployment still faces hallucinations, outdated knowledge, and privacy risks. Retrieval‑Augmented Generation (RAG) combines a retrieval engine with a generative model to mitigate these problems by injecting external, up‑to‑date knowledge into the generation process.

The core RAG system consists of several components: data sources, preprocessing modules, a retriever, a ranker, and a generator. A modular architecture separates traditional RAG (indexing → retrieval → generation) from advanced RAG, which adds query rewriting, HyDE‑based document generation, post‑retrieval re‑ranking, and filtering before feeding the final context to the LLM.

In the production‑grade design, three hierarchical layers are employed. The bottom layer hosts algorithmic modules (OCR, multi‑turn query rewriting, tokenization, table recognition). The middle layer implements offline indexing (document parsing, tokenization, vector creation) and online QA (query rewriting, hybrid retrieval, ranking, LLM generation) using vector databases, Elasticsearch, and MySQL. The top layer provides user‑level configuration for knowledge bases, model selection, and dialogue rules.

Key engineering challenges are addressed in three stages: “search more comprehensively,” “rank better,” and “answer more accurately.” Comprehensive search is achieved through OCR‑based document parsing, multi‑turn query rewriting, and hybrid retrieval that fuses dense vector similarity with BM25 keyword matching. Better ranking combines a coarse RRF‑based fusion of multiple recall lists, a ColBERT token‑level ranker, and a cross‑encoder re‑ranker, followed by knowledge‑filtering via an NLI classifier. Accurate answering relies on knowledge formatting, prompt templating, and a two‑stage generation (outline → final answer) using FoRAG to improve factuality and logical consistency.

The final summary emphasizes that building a robust RAG system requires careful attention to each pipeline stage—data ingestion, retrieval, ranking, and generation—rather than relying solely on the LLM, and that a modular, horizontally scalable design enables continuous optimization for enterprise use cases.