From Agent Error to Team Capability: A Systematic Improvement Framework

Fix the Immediate Issue, Then Trace the Root Cause

When using AI for development, teams often focus only on correcting the current error. The more valuable question is whether the same problem will recur on the next task or with a different developer. If experienced people must repeatedly intervene, the team remains dependent on a few individuals despite growing code volume.

The author references an InfoQ summary of Patrick Debois's talk ( "Stop Fixing Code, Fix the System: DevOps Father Says Organizational Change in the Agent Era Is Harder Than Technology" , 2026-08-06) which emphasizes improving the system that supports the agent, not just the agent's output.

The real step forward is asking: why did this require human correction, and how can we make the next occurrence avoid the same detour?

Role Separation to Pinpoint Responsibility

The author divides work into three roles: Architecture (maintains requirements and design), Development (implements per design), and Testing (verifies design and implementation, feeds back issues). This separation does not require multiple sessions for every change; small, well-scoped fixes can be done and self-tested by Development. When design gaps appear, Development proposes changes, Architecture confirms and updates documents, and Development does not rewrite requirements or designs independently. Blocked work waits; unaffected work continues.

Key results must still be verified against confirmed designs and acceptance criteria, combining human review, independent verification roles, and automated tests that Development cannot weaken. A separate session does not guarantee finding different errors, and passing tests does not prove business assumptions are correct.

The value of this division is that problems can be routed to the role with authority to decide. However, if the author personally shuttles messages between roles, experience stays trapped in chat logs.

Hypothetical Case: Batch Import Retry Duplicates

An agent builds a bulk-import feature for a management system. The design states "support retry on failure" but does not specify handling when some records have already been written. After implementation, testing discovers that retrying re-creates records.

The fix cannot be merely "add a check to avoid duplicate inserts." First, the business must clarify allowed outcomes: all-or-nothing batch, or partial success? Retry the whole file or only failed items? By what key is a business record identified?

Assume the project decides: partial success allowed; retry the whole file but never re-write already successful records; only reprocess failed items. Architecture then adds record identification, status, and interface contracts; Development corrects the implementation; Testing verifies both first-run failure and subsequent retry results.

After the feature works, the team must check: do other import tasks lack similar contracts? Why did design review miss this? Why did existing tests not cover it? This is where team capability improvement begins.

Same Symptom, Different Improvement Points

"Agent did poorly" is not a specific cause. One must investigate both why the error occurred and why existing checks did not catch it. The same duplicate-write symptom can reveal failures at different stages.

Design omitted partial-success and retry rules → Supplement requirements and design; get business confirmation on handling rules.

Design was clear but agent read an outdated version → Fix design entry points, version identifiers, and task references.

Agent read valid design but implementation still deviated → Correct code; verify that task constraints feed into implementation and review.

Tests covered only the happy path → Add cases for partial failure, duplicate requests, and post-recovery state checks.

Test command exited successfully but key cases were not executed → Fix test entry points and reports; make actual execution scope explicit.

Judgment must be evidence-based: which design version was used, which files changed, which checks actually ran — these records are more useful than "model not smart enough." Multiple causes can coexist. If the model has full context yet still makes reasoning or implementation errors, compare task decomposition, model configuration, or tooling rather than endlessly adding prompt text.

Adding a prompt hint is only one candidate fix. First confirm which stage failed, then decide which artifact to change.

Turning One Reminder into a Reusable Mechanism

Continuing the import example, assume the main gaps were: design phase lacked explicit failure/retry contracts, and tests covered only the happy path.

Lightweight template improvement : Add required questions to design templates for similar import tasks: how to identify business records, whether partial success is allowed, where retry starts, how to judge correctness. Templates prompt humans and agents to ask; specific handling still needs business confirmation.

Executable verification : Build a test set with records that partially fail, execute, then retry; verify successful items are not duplicated and failed items follow the contract; retain version and results. This can be packaged as a test helper for similar projects. The reusable part is the ability to construct failures, execute retries, and check state; concrete objects, fields, and business judgments remain project-specific.

Enforcement at execution entry : Record the design version supplied to the task; pipeline verifies the tests actually run. Version records only prove what was provided; whether implementation honors contracts still relies on review and test checks.

No complex platform is needed initially. One template, one executable test set, one clear maintainer — these can move experience out of private conversations. The key is that the next task actually uses them.

Validate Reusability Before Sharing

Placing a rule in a shared directory is easy; confirming it will not mislead other projects requires more rigor.

Using the retry check as an example: some imports allow partial success, others require full rollback. If a shared component hard-codes one business choice, it may reduce issues in the current project but introduce errors elsewhere.

Therefore, shared improvements must document: applicable scope, configurable parts, unsupported scenarios, and a designated maintainer. After updates, validate with target scenarios and retain counterexamples for different handling modes to confirm the component does not overstep its boundaries.

Shared components also need versioning and change logs. After one project validates, adopt in a few applicable tasks first, then expand based on results. If quality drops or maintenance cost rises sharply, be able to halt rollout or revert to the validated version.

Public capabilities need a maintainer; project teams own business applicability decisions. In small teams the same person may wear both hats, but unified tooling must not eliminate applicability judgment.

Reduce Manual Intervention, But Measure What Actually Decreased

Improvements should cut repetitive work like re-explaining valid rules or hunting for test entry points. Clarifying ambiguous requirements, deciding failure handling, and approving high-risk actions may still need human decisions.

If prompt count drops but false positives, rework, and maintenance burden rise, the improvement is not effective. Human intervention must be evaluated alongside result quality and total investment; specific recording and comparison methods will be covered in a follow-up article.

Allocate Time for Improvement and Define Exit Criteria

These activities do not happen automatically because the team bought AI tools. The team must assign owners for collecting recurring issues, maintaining shared capabilities, and verifying improvement effectiveness.

During retrospectives, pick one recurring, high-impact problem; bring task records and failure evidence; define a small-scope improvement. After the owner completes it, use subsequent tasks to check whether similar rework decreased, then decide whether to keep and expand.

Not every error warrants a shared rule. One-off typos can be fixed directly; project-specific business conventions stay in the project. High-impact issues even if seen once may need immediate controls; high-frequency low-impact issues require comparing automation investment against benefit.

Regularly retire obsolete guidelines and duplicate implementations, confirming no project still depends on them. Capabilities now handled by tools need not be sustained by repeated reminders; but still-valid business, security, and acceptance constraints must be confirmed as covered by existing mechanisms before retiring old guides.

The outcome of continuous improvement should be a team that more easily gets work right — not thicker shared documents and ever-growing checklists.

Summary

After an agent error, fix the current code, but also examine the conditions that let the error recur. The goal is for the issue an experienced person caught this time to become part of effective design, executable checks, or maintained tools so that others benefit next time. That way, experience stops living only in a few people's repeated rescues.

Reference

InfoQ, Fu Yuqi & Tina: "Stop Fixing Code, Fix the System: DevOps Father Says Organizational Change in the Agent Era Is Harder Than Technology" , 2026-08-06. https://www.infoq.cn/article/hLA2I6DD1v0ou0sE8KKB This article uses the Patrick Debois talk summarized there as a starting point. The batch-import correction process is a hypothetical case; problem localization, sharing validation, and metric trade-offs are the author's analysis and do not represent completed team effectiveness evaluations.