How Agents Evolve Without Degrading: From Risk Control to Semantic Engineering

The July 2 live session brought together host Xiang Qiaorui and two guests—Li Qin, a veteran of financial risk and credit automation, and Zhao Heng, focused on data‑engineering agents—to explore ten core questions about building, evolving, and cost‑controlling agent systems, emphasizing engineering realities over hype.

All three stressed that enterprises need agents that are stable, explainable, auditable, and continuously optimizable; evolution means sustainable improvement rather than making agents more human‑like, and preventing degradation means preserving clear responsibility boundaries.

Li Qin highlighted that in banking the model must not sit at the decision front‑line. Large models should serve as complex reasoners and evidence organizers, while rule engines, fine‑tuned small models, and other deterministic modules enforce the bottom line—"small models guard the bottom line, large models raise the ceiling."

Xiang added a platform perspective: high‑certainty, rule‑clear steps use workflows and a mix of small and large models; more exploratory scenarios such as dynamic analysis, attribution, or news understanding employ ReAct‑style agents that autonomously call tools. The underlying data, permission, and semantic layers are unified, keeping the top‑level agent lightweight and exposing atomic tools.

Zhao observed that most agent architectures converge to a set of tools, prompts/skills, and behavior norms. The differentiator lies in designing verifiable sub‑task structures and native function‑calling, thereby reducing reliance on heavyweight orchestration.

Regarding abandoning routes, Li Qin recounted early attempts to let large models directly approve credit, which quickly failed due to uncontrolled latency, hallucinations, and insufficient explainability. The team shifted the large model’s role to "assistant reasoning" and moved away from a monolithic single‑agent design toward multiple agents with clear business boundaries.

Evaluation should go beyond a single correct answer. In finance, metrics such as approval‑rate vs. bad‑debt rate, collection‑recovery improvement, complaint reduction, and human‑review stability are used to gauge net business gain. Xiang introduced the engineering metric "trajectory length"—long token‑heavy chains indicate poor scalability. Zhao stressed the need for benchmark sets and validation loops to pinpoint failures in model, context, tool use, or verification.

Stability, not functionality, is the main gap from "usable" to "good." Li Qin noted error amplification in multi‑step tasks, while Zhao highlighted user adoption challenges and the necessity of semantic consistency and versioned metrics. All three underscored Human‑in‑the‑Loop as essential because responsibility cannot be fully delegated to AI.

Cost control follows a tiered strategy: cheap, high‑frequency, rule‑clear steps use small models or rule systems; complex reasoning, evidence stitching, and multimodal analysis use large models. Zhao warned that overall cost spikes when execution paths are long or context handling is poor; atomic tools reduce token usage and time.

Scaling from pilot to replication introduces new problems: maintaining performance across different organizations, data silos, compliance constraints, and coordination among many agents. Organizational resistance and trust issues become as challenging as technical ones.

Looking ahead, the panel sees the biggest opportunity in engineering maturity—better memory, skill abstraction, and robust benchmarks—while the biggest uncertainty lies in regulation, consumer appeals, and whether companies view AI as augmentation or replacement.