Ensuring Stable AI Agents: Engineering Practices, RAG, and Monitoring

Background

With the rapid development of large‑model technology, many LLM‑based applications such as AIGC image generation, chatbots, and automatic document processing have emerged. However, stability problems become prominent in complex scenarios, including hallucinations, knowledge quality, and retry/exception handling.

Agent Overview

AI Agent is an intelligent entity that perceives the environment, makes decisions and executes actions. Unlike traditional AI, an Agent can plan and use tools to achieve goals, e.g., processing a refund by confirming intent, gathering necessary information, and invoking the refund system automatically.

Key modules: prompt , Chain , LLM , Tools , Actions .

Stability Issues in Customer‑Service Agent

Business context: e‑commerce chatbot with a QA‑style knowledge base.

Hallucination problems: caused by insufficient or low‑quality training data, model complexity, and language ambiguity.

RAG (Retrieval‑Augmented Generation) can reduce hallucinations by about 80 % and, with appropriate prompting, prevent the model from fabricating answers when no knowledge is retrieved.

Memory supplementation: injecting user‑specific information (e.g., membership level) into the prompt improves answer accuracy.

Engineering Optimizations

Query rewriting to improve RAG recall.

Handling dirty or stale knowledge‑base data.

Improving corpus quality by converting card‑style data into natural language.

Ensuring consistent JSON output formats; providing few‑shot examples and exception‑handling pipelines for malformed responses.

Exception Handling and Retry Strategies

Define retryable exceptions (e.g., RPC/HSF failures, short‑duration HTTP errors) and non‑retryable ones (high‑cost retries, idempotent tools, unhandleable errors). Control ReAct loop iterations and set time limits to avoid infinite loops.

Human‑in‑the‑Loop Guardrails

When the Agent cannot resolve an issue, it should signal hand‑over to a human operator, especially for abnormal cases, uncovered scenarios, or multimodal inputs.

Monitoring

Monitoring focuses on critical points: LLM invocation metrics, tool‑call latency, and RAG performance. Visual dashboards illustrate latency distributions and error rates.

Code Examples

user: 我的XX卡折扣是多少？
aiAssistant: ?

user: 我的XX卡折扣是多少？
aiAssistant: 您好，我们的会员卡对于不同会员折扣不同，初级会员是5折，.....

user: 我买了个糕点吃不完了
aiAssistant: 您好，您是买的哪一款糕点，有什么问题呢？
user: 保质期多久呢

user: 我买了个糕点吃不完了
aiAssistant: 您好，您是买的哪一款糕点，有什么问题呢？
user: 糕点的保质期有多久呢？