From Risk Control to Semantics: How Agents Self‑Evolve Without Degrading

On July 2, a live broadcast titled “Harness Engineering and Agent Practice—From Single Agent Failure to Multi‑Agent Collaboration” brought together host Xiang Qiaorui with guests Li Qin (financial risk‑control) and Zhao Heng (data‑engineering) to discuss ten practical questions about building, scaling, and governing AI agent systems.

1. First‑principles architecture: keep large models off the front line

Li Qin emphasized that banking workflows demand rigor, auditability, and determinism, while large language models are inherently nondeterministic. Their architecture therefore assigns small, fine‑tuned models or rule engines to deterministic steps and reserves large models for complex reasoning and evidence‑chain organization—"small models guard the bottom line, large models lift the ceiling."

2. Modular, skill‑driven design

Host Xiang noted that different customer scenarios require different recommendations. High‑certainty tasks such as dynamic analysis, attribution, or financial‑report understanding are handled by ReAct‑style agents that autonomously call tools, while stable, rule‑driven tasks remain in workflow‑rule chains. Underlying data, permissions, and semantics are unified in a single layer, keeping the agent layer lightweight.

3. Embracing multi‑agent over single‑agent

Early attempts to let a single agent cover fraud detection, credit scoring, collection, and post‑loan monitoring ran into latency, hallucination, and explainability problems. The team switched to a multi‑agent architecture where each agent owns a clear business boundary, and a higher‑level coordinator resolves conflicts.

4. Evaluation beyond correctness

In finance, Li Qin measures success by business metrics such as approval‑rate vs. bad‑debt rate, collection recovery, complaint reduction, and stability of human‑review pass‑rates. Xiang added an engineering metric—trajectory length—stating that long token‑heavy chains indicate poor scalability even if the final answer is correct. Zhao stressed the need for benchmark suites to pinpoint failures in model, context, tool‑calling, or validation stages.

5. Stability versus functionality

Multi‑step tasks suffer exponential error amplification; errors often appear linguistically reasonable, making them hard to reproduce. Prompt sensitivity is described as a "black art" that is unacceptable in high‑risk domains.

6. Cost control through waste reduction

Li Qin’s tiered strategy assigns high‑frequency, deterministic steps to small models or rule systems, reserving large models for complex reasoning, evidence aggregation, and cross‑modal analysis. Zhao warned that overall cost spikes when execution traces become long, context organization is poor, or unnecessary calls proliferate. Xiang advocated atomizing capabilities into fine‑grained tools, which reduces token usage, latency, and failure cost.

7. Human‑in‑the‑loop and responsibility boundaries

All three speakers agreed that decisions carrying legal or ethical consequences must retain a human in the loop. Agents can provide evidence, suggestions, and risk alerts, but final liability rests with people; the boundary is defined by responsibility, not by intelligence.

8. Organizational friction and trust

Employees may fear knowledge distillation and job displacement. Zhao highlighted that acceptance hinges on viewing AI as augmentation rather than replacement; without trust, even the best technical solution struggles to scale.

9. Scaling from pilot to enterprise

When moving from single‑point pilots to enterprise‑wide replication, the focus shifts from "can the agent run?" to "can it maintain performance across teams, data silos, and compliance regimes?" Zhao noted that increasing the number of agents amplifies coordination complexity, boundary ambiguity, and conflict, while also raising employee concerns about AI replacing their work.

10. Future outlook (1‑2 years)

The panel is cautiously optimistic. Reinforcement‑learning closed‑loops, private small models, and multimodal capabilities will deepen agent integration, especially in data‑engineered, risk‑controlled domains. However, regulatory definitions of AI decision boundaries, consumer appeal processes, and organizational willingness to treat AI as an empowering tool are the biggest variables affecting adoption speed.

Overall, the discussion stresses that true agent evolution is not a single leap in capability but a continuous, accountable engineering effort that balances stability, cost, and responsibility while gradually improving decision quality.