How RAG is Shaping the Future of AI-Powered User Experience

Background

Since OpenAI released ChatGPT‑3.5 on 2022‑11‑30, pre‑trained large models have grown exponentially. By March–April 2023 domestic leaders such as Alibaba’s Tongyi Qianwen and Baidu’s Wenxin Yiyan launched their own models, marking the start of a model‑driven AI era. Open‑source releases like DeepSeek‑R1 and QwQ‑32B (Jan‑Mar 2025) further accelerated adoption across finance, healthcare, education, and especially intelligent Q&A services, where large models have dramatically improved user experience.

However, deeper integration raises challenges: vertical data alignment, hallucinations, and the need for efficient user‑experience optimization. Retrieval‑Augmented Generation (RAG) emerges as a promising solution.

RAG Development Trends

Sequoia Capital predicts three 2025 AI trends: (1) differentiation among LLM providers (OpenAI, Amazon/Anthropic, Google, Meta) with varied business models; (2) AI‑search becoming a killer application, shifting from keyword indexing to semantic understanding; (3) ROI pressure demanding tangible commercial outcomes.

Market research (Menlo Partners, 2024‑11‑20) shows RAG adoption rising from 31% to 51% among enterprises, while fine‑tuning and RLHF remain low (9% and 5%). The upcoming inference boom (DeepSeek‑R1, QwQ‑32B) is expected to further boost RAG relevance.

Technical Challenges

Business Challenges

Deterministic replies and lack of generative thinking degrade user experience.

High maintenance cost for expert knowledge; inconsistent answers across agents.

Answers often generic, missing context, leading to user frustration.

Goals: (1) Deliver professional, precise, human‑like service; (2) Provide agents with comprehensive AI assistance.

Core Technical Challenges

Data Value Dimension – "Garbage in, garbage out" limits RAG. Heterogeneous data (documents, notes, internal memos) must be cleaned, core information identified, and relationships mapped.

Heterogeneous Retrieval – Traditional vector similarity cannot handle multi‑source, multi‑style data. Precise Q‑A mapping (single‑/multi‑hop) is required.

Generation Control – Simple context injection leads to hallucinations. Retrieval results must become trustworthy anchors, filtering out irrelevant content.

Evaluation System – Conventional metrics (BLEU, ROUGE) miss RAG‑specific defects. A multi‑dimensional dynamic matrix is needed.

Empirical data: retrieval accuracy reached 83% but generation accuracy only 66%, creating a 17‑point trust gap.

Overall Solution

1. Data Value Breakthrough

Initial chunking based on document hierarchy proved insufficient. The next phase builds a layered knowledge graph, enabling hierarchical retrieval (text → semantic → logical) and supporting multimodal formats.

2. Heterogeneous Retrieval Leap

We moved from a single vector space to a hybrid "semantic + vector + graph" architecture, adding iterative retrieval for complex queries. Key steps:

Problem Understanding – Decompose complex questions, apply multi‑strategy pipelines, and avoid over‑splitting.

Retrieval Strategy – Mix text, vector, and graph recall; introduce iterative refinement.

Results: Top‑5 recall improved by 4 pts; multi‑semantic + graph recall boosted Top‑5 accuracy markedly, while Top‑10 remained stable.

3. Generation Control Optimization

Two pillars:

Filter Invalid References – Ensure retrieved content is both relevant and capable of answering the query.

Enhance Reference Quality – Leverage longer context (small‑to‑big strategy), assign structured IDs for fast lookup, and enrich image references with summaries and descriptions.

Performance gains: rewrite pipeline reduced latency from 4.5 s to 1.5 s (>60% faster); high‑score knowledge bypasses rewrite, further cutting time.

Human‑labeled GSB (Good/Same/Bad) evaluation showed Good : Same : Bad = 61.51 % : 26.36 % : 12.13 % for the new rewrite approach.

4. Evaluation System Reconstruction (RAG Diagnoser)

We introduced a fine‑grained diagnostic framework:

Classify user queries (e.g., Factual, Analytical, Comparative, Tutorial).

Extract atomic facts from ground‑truth and model output.

Measure fact‑level precision, recall, contradiction, and coverage across retrieval, rewrite, and generation stages.

Case study: a query about "g8i server second‑level virtualization" revealed contradictory facts and missing key information, guiding targeted improvements (expand synonym library, adjust rewrite logic).

Future Outlook

Multimodal Retrieval – Fuse text, tables, and images for richer understanding.

DeepSearch – Build deeper heterogeneous graphs and embed long‑term reasoning into the search pipeline.

Comprehensive Evaluation – Refine the RAG Diagnoser to drive continuous iteration.

With rapid advances in inference models, the next phase (DeepRAG) aims to deliver more reliable, intelligent services.

References

[1] https://arxiv.org/pdf/2503.09567

[2] https://mp.weixin.qq.com/s/KmDFqJJbJjsZm8sV28lg2g

[3] https://menlovc.com/2024-the-state-of-generative-ai-in-the-enterprise

[4] https://arxiv.org/abs/2410.12248

[5] https://github.com/amazon-science/RAGChecker

[6] https://github.com/explodinggradients/ragas

{
  "QuerySourceTicket": "xxx",
  "UserQuestion": "g8i服务器支持二次虚拟化吗",
  "QueryCategory": "Factual-Y/N",
  "GroundTruth": "不是的。仅弹性裸金属服务器和超级计算集群支持二次虚拟化，g8i服务器属于通用型实例规格族，不支持二次虚拟化。",
  "ModelAnswer": "g8i服务器属于弹性裸金属服务器，支持二次虚拟化。",
  "Analysis": {
    "Contradiction": "模型回答中‘支持二次虚拟化’与事实相冲突",
    "MissingFacts": ["弹性裸金属服务器支持二次虚拟化", "超级计算集群支持二次虚拟化"]
  }
}