Why AI Agents Fail: The Three‑Layer Harness, Loop, and Graph Architecture

Introduction

AI agents often appear to fail because the model is not smart enough, but the real cause usually lies in the surrounding system. Two teams using the exact same model on identical tasks produced dramatically different outcomes—one succeeded in minutes, the other failed catastrophically—showing that the model is not the primary variable.

Harness: What the AI Is Allowed to Do

Harness engineering defines the tools, files, memory, and permission rules an AI can access. A raw model cannot open files, run tests, click in a browser, or retain memory; these capabilities are added through the harness. Teams that give the model a clean workspace and clear boundaries see reliable results, whereas teams that provide vague instructions and broken tools encounter failures. Anthropic’s experience building a multi‑session coding assistant illustrates this: early attempts to summarize old conversations for space savings failed, leading them to adopt an onboarding‑style harness with a project‑initialization file, an evolving progress log, and clean notes for each session.

Loop: How Work Is Checked

Loop engineering designs the iterative process of trying, observing results, and retrying. A good loop includes a clear goal, a test to verify achievement, honest feedback on failure, and a stop condition. The author warns against letting the AI decide it is finished; confidence alone is insufficient evidence—only passing tests or external verification counts.

Typical loop patterns include:

Check‑and‑retry loop: AI reworks until it passes a real test.

Wake‑up loop: AI activates only on specific triggers (e.g., new email, scheduled time).

Improvement loop: Reviews recent failures and silently rewrites prompts to avoid repeat errors.

Each additional check adds time and compute cost, so loops should be added when the cost of errors exceeds the cost of verification.

Graph: Mapping Subsequent Actions

Graph engineering answers “what can happen next and in what order.” It resembles a flowchart where nodes are steps and edges define permissible sequences, parallelism, convergence, or human‑in‑the‑loop approvals. For example, an AI system generating a report might have separate stages for fact collection, fact‑checking, drafting, and human review, with the graph enforcing that drafting cannot start before fact‑checking completes and that publishing requires approval.

Graphs add value when real branches, approvals, or handoffs exist; otherwise they may be unnecessary overhead.

Why It Matters Even If You Don't Build Such Systems

The author reflects that many perceived AI product improvements are actually better harness, loop, or graph designs rather than smarter models. Users may notice differences between a coding assistant that silently validates its work and one that delivers broken code, or between a support chatbot that remembers recent context and one that forces repetition. The underlying model is often the same; reliability hinges on the surrounding engineering layers.

Five Ways Teams Unintentionally Break It

Prematurely drawing a detailed flowchart before observing the AI’s actual behavior, leading to mismatched steps.

Having the AI self‑grade its work, which repeats the same blind spots; external tests are needed.

Creating an unrestricted retry loop without clear termination, causing endless costly cycles.

Treating the harness as a junk drawer—adding more tools and memory indiscriminately, increasing the chance of selecting the wrong tool.

Blaming the AI when surrounding processes are broken—outdated notes, unclear instructions, or missing stop rules cause even a smarter model to repeat mistakes.

A 60‑Second Checklist for Any AI Tool

When an AI tool disappoints, run through the three layers before assuming the model is at fault: verify the harness (tool access, permissions), examine the loops (verification and stop conditions), and inspect the graph (workflow ordering and approvals).

Key Takeaways

The model is the talent; harness, loop, and graph constitute the work. A capable new employee without a laptop, manager, or org chart will fail, just as an AI agent will fail without proper harness, loop, and graph. Diagnosing which layer is broken is the first step to fixing unreliable AI systems.

References

Anthropic, "Building Effective AI Agents"

LangChain, "The Anatomy of an Agent Harness"

LangChain, "The Art of Loop Engineering"

LangChain, "LangChain and LangGraph Reach Their v1.0 Milestones"

OpenAI, "Agents SDK Guide"

OpenAI, "A Practical Guide to Building Agents"

Microsoft, "GraphFlow (Workflows), AutoGen Documentation"

Microsoft Research, "Introducing AutoGen Studio"