Five Gates to Production-Ready Financial Agents: DeWu's Reliability Framework

Problem: From Demo to Production

Agents that understand requirements, call tools, and generate results look impressive in demos but often fail in production: data may be unreliable, rules may drift, and retries may cause duplicate execution. The production challenge is not "can it do the task" but "why does it do it" and "what happens when it goes wrong".

Core Insight: Production-Grade Agent as a Manageable System

A production-grade agent is not a smarter model but a system that can be managed, verified, traced, and recovered. The bottleneck lies not in model capability but in the absence of runnable mechanisms for manageability, verifiability, traceability, and recoverability.

Solution: The Five Gates Framework

Evidence Gate : Conclusions must be traceable to data sources, metric versions, and business rules.

Determinism Gate : LLMs handle understanding and interpretation; code handles SQL, formulas, thresholds, and state judgments.

State Gate : A lifecycle of Candidate → Shadow → Certified → Degraded → Blocked.

Runtime Gate : Idempotency, retry, read-back verification, anomaly monitoring, and rollback.

Responsibility Gate : The agent drives the process, while humans retain confirmation of calibers, major exceptions, and final approval authority.

Validation: Shadow Acceptance Testing

Shadow acceptance runs with real data and real business rhythm but produces no real business consequences. It verifies correctness, evidence completeness, anomaly stop behavior, that re-runs do not create duplicate actions, and that the system can recover and roll back. This method suits zero-error-tolerance scenarios such as financial calculations.

Before-and-After Comparison

Capability determination shifts from "ran once equals usable" to requiring passage through Candidate, Shadow, and Certified states with degradation and blocking capabilities. The division of labor changes from the LLM handling everything to: LLM for understanding and interpretation, code for deterministic execution, and humans for major decisions.

Key Principle

A reliable agent is not one that does everything, but one that knows when it should not continue.

Process Restructuring

Do not place a chatbot at the front of the process. Instead, restructure as: Agent understands requirements → code completes deterministic processing → Agent identifies anomalies → humans handle exceptions → system drives approval and review. Redefine who sets rules, who handles exceptions, who approves, and who is responsible.

Practical Takeaways

A five-gate checklist (Evidence, Determinism, State, Runtime, Responsibility) for pre-launch self-assessment.

A shadow acceptance method using real data and rhythm without real consequences, focusing on correctness, evidence, anomaly stop, no duplicate actions on re-run, and recoverability.

A process restructuring mindset that separates understanding, deterministic execution, anomaly detection, human exception handling, and system-driven approval.