What I Learned After Six Months Building Production AI Agents: The Five Costly Mistakes

Problem Statement

AI agents that look impressive in demo environments often fail in production with silent errors, prompt regressions, tool‑call loops, uncontrolled cost, and hard‑to‑trace exceptions. The root cause identified after six months of building a ticket‑routing assistant was the absence of a structured test harness that makes the agent observable, controllable, and evolvable.

Five Common Mistakes

Treating the prompt as the whole product. Prompt changes were made without automated regression checks, so regressions went unnoticed until users reported them.

Relying on manual inspection. Spot‑checks missed subtle regressions and edge‑case failures; issues persisted for weeks before being discovered.

Ignoring cost and latency as first‑class constraints. The agent sometimes called expensive models or entered tool‑call loops, burning tokens and increasing latency without any budget alerts.

Underestimating tool and API fragility. Rate limits, outages, and schema changes caused silent failures; no fault‑injection tests verified graceful degradation.

Assuming the model will never misbehave. Without explicit safety policies the agent produced undesirable outputs, such as fabricating a non‑existent company policy.

Harness Engineering

Borrowing from software test harnesses, Harness Engineering wraps an agent in a structured environment that makes it testable, observable, constrained, and evolvable. It consists of five layers, each building on the previous one.

Layer 1 – Scenario & Dataset Design

Collect real production inputs (30‑50 examples) or construct realistic synthetic ones.

Label the expected output for each input.

Define simple scoring functions (e.g., presence of a required field, completion within N steps).

Ensure the dataset covers three categories:

CommonCases – typical daily inputs.

EdgeCases – malformed formats, ambiguous phrasing, incomplete information.

AdversarialCases – inputs designed to trigger policy violations or undesirable behavior.

Example: In production, ~15 % of messages contained stray HTML tags or multilingual content that were absent from the original 40‑case test set, leading to silent mis‑routing of tickets for three weeks before a manager noticed the pattern.

Layer 2 – Evaluation Harness

Run the agent on the scenario dataset, compute scores, and store the results as a baseline. Subsequent runs compare scores against the baseline to detect regressions automatically.

Concrete benefit: a regression that previously required hours of manual debugging was identified in seconds after the harness was added.

Layer 3 – Cost & Latency Harness

Instrument the agent to log token usage and execution time for every call.

Configure budget thresholds (e.g., maximum tokens per request, P95 latency limit).

Raise alerts when thresholds are exceeded in development, CI, or production.

Observed issues: an agent used a costly large model for tasks that could be solved with a cheaper model, and occasional tool‑call loops increased the number of rounds from the expected 2‑3 to 6‑7, inflating token consumption without adding value.

Layer 4 – Safety & Policy Harness

Define testable guardrails (what the system must do and must not do).

Version‑control the policy suite.

Run red‑team simulations regularly (e.g., every Friday) to inject adversarial inputs and verify that guardrails hold.

Without this layer, the agent once fabricated a non‑existent company policy that was later quoted verbatim by a support representative.

Layer 5 – Observability & Feedback Harness

Emit a trace for every agent invocation: input, tool calls, per‑step timings, final output, and user feedback flag.

Store traces in a searchable log store.

Feed trace‑derived insights back into the scenario dataset and cost/latency monitors.

Traces turn production debugging from an archaeological dig into a simple query.

Four‑Week Practical Adoption Guide

Week 1 – Define scenarios and success metrics. Collect 30‑50 real inputs, label expected outputs, and implement 2‑3 heuristic scorers (e.g., field presence, N‑step completion).

Week 2 – Build a minimal Evaluation Harness and establish a baseline. Write a script that runs the agent on the scenario set, prints scores, and saves the baseline for future comparison.

Week 3 – Add cost, latency, and basic safety checks. Log token counts and latency, set rough budget limits, and add 3‑5 policy tests targeting high‑impact failures.

Week 4 – Integrate the harness into the release process. Hook the Evaluation Harness into CI so that no change can be merged without passing the harness; schedule regular production‑trace reviews to keep the dataset fresh.

Key Takeaways

Continuous observability, automated regression testing, cost monitoring, and safety validation transformed a flaky demo into a production‑ready service. Teams should audit which of the five layers they already have and prioritize filling the gaps before scaling agents further.

Code example

作者：Sudip P.
翻译：林珑
校对：
陈超
本文
约6400字
，建议阅读
15
分钟
给那些厌倦了“Demo一切正常，一上线就翻车”的资深工程师的一些真心话。