General Agent Harness Architecture: From 4 LLM Defects to 36 Functional Modules

1. Agent = Model + Harness

The article opens with the LangChain formula: Agent = Model + Harness . In the first year after ChatGPT, models were used only for single-turn Q&A since 2024, coding agents (Codex, Claude Code, DeepSeek dsh, Kiro) and deep-research products let the model read files, run commands, and iterate autonomously. The model itself did not change — the surrounding layer (Harness) did.

Two interpretations:

Qualitative: Harness fills Model's gaps. A model is a stateless text function (input text → probability sampling → output text) with no "hands"; everything that compensates for this is Harness Engineering.

Quantitative: The agent's goal decides which gaps to fill and how thickly. Different verticals need different subsets and thicknesses.

2. Model's Nature and Four "Cannot-Dos"

2.1 Premise: Model is a stateless text function

One call takes bounded text, samples from a distribution, emits bounded text, and is independent of all other calls.

2.2 Premise: Agent goal is to reliably complete a real task

A task is a constrained state trajectory: from current state, through a series of actions, to a goal state satisfying constraints. Therefore an agent needs four necessary-and-sufficient conditions (ReAct's "reasoning" is done by the model, so omitted):

Perception — know current state and relevant facts.

Action — effect changes on the external environment.

Continuity — progress across steps is not lost.

Correctness — actions and results match goal and constraints.

Necessity: drop any one and the task fails. Sufficiency: the four aspects are exactly the decomposition of the trajectory definition.

2.3 Conclusion: Harness must solve Model's four cannot-dos

It knows nothing outside the input (perception gap): only sees the prompt; latest facts, private data, system state, remaining quota are absent. Input length is limited, so not everything fits, and credibility of included parts is indistinguishable.

It cannot act (action gap): output is text, not action; external code must interpret it. Output length is limited, so long tasks need multiple steps.

It does not remember the last call (continuity gap): each output depends only on current input; prior conclusions, outputs, and authorizations do not carry over.

It does not guarantee 100% correctness and is unaware of its errors (correctness gap): answers may be wrong, outputs contain no reliable claim about factual gaps, so it confidently states falsehoods and can be manipulated by injected content.

3. From Needs to Responsibilities

A mapping from each agent need to a solution method and a Harness responsibility:

Agent Need: Unknown outside input<br/>

Solution Method: Write external facts into prompt before each call; design context-space utilization due to length limit; mark provenance of each segment.

Harness Responsibility: Context Construction

Agent Need: Cannot act<br/>

Solution Method: External program interprets model output as actions and executes; support repeated calls because one turn cannot finish.

Harness Responsibility: Action (Tool Loop)

Agent Need: Does not remember last call<br/>

Solution Method: Persist facts needed for next call in session outside model; append-only so original records are recoverable.

Harness Responsibility: State (Session, Task Progress)

Agent Need: Not guaranteed correct (pre-action)<br/>

Solution Method: Before action, judge by rules: allow, deny, or escalate for approval.

Harness Responsibility: Action Control (Pre-action)

Agent Need: Not guaranteed correct (post-action)<br/>

Solution Method: After action, judge by checkers: retry, rollback, or escalate.

Harness Responsibility: Verification (Post-action)

Agent Need: User interface gateway<br/>

Solution Method: Single entry point for inbound requests, outbound responses, and approval escalation.

Harness Responsibility: Access

Agent Need: Scenario extension & generalization<br/>

Solution Method: Declare policies/rules (config, permissions), extensions (plugins, MCP, Skills, system prompts), and hook mount points as runtime-readable data, layered loading — change behavior without code changes.

Harness Responsibility: Policy & Extension

Agent Need: Observability, audit, evaluation, optimization<br/>

Solution Method: Write-only recording of execution process, rule hits, token usage, decision bases for later audit, evaluation, optimization.

Harness Responsibility: Observation

Key note: "Not guaranteed correct" splits into pre-action (rules) and post-action (checkers) because action splits the timeline; the two judgments cannot merge.

4. Eight Responsibility Domains and 36 Functional Modules

Each responsibility becomes a domain; domains are the implementation units. The eight domains and their modules:

4.1 Context Domain

Fills "doesn't know outside chat window". LLM only sees inside the window; must feed needed info in while controlling window size to prevent context rot. Core module: Context Construction (typical tech: RAG — retrieve from knowledge base, assemble into context, include task goal, role, acceptance criteria). Also manage window size: attention budget is limited; key info in middle of long context has lower recall probability. Techniques: progressive disclosure (give index first, fetch on demand), compress near limit (summarize history in-place), offload intermediate artifacts to files.

Context Construction — Assemble scattered facts from files, retrieval, history into this input; record provenance for debugging. Codex splits input into ~43 segments.

User Reference Parsing — Turn @file references into actual content. Claude Code's file reference syntax; Kiro's context reference syntax.

Project Rules Loading — Auto-include project conventions and role definitions each turn. Claude Code's CLAUDE.md.

Context Window Quota — Track each segment's token usage and remaining budget; compression decisions driven by quota, reserving buffer for compression itself. Claude Code shows per-block usage and reserved buffer.

Context Compression — When near limit, compress old history to summary and continue. Claude Code auto-compresses near limit.

4.2 Action Domain

Fills "cannot act". Example: "Why does K8s service return 502?" Without this layer, LLM only recites "502 usually gateway issue." With it: runs kubectl get pods, sees CrashLoopBackOff, runs kubectl logs, sees connection refused: redis:6379, follows lead. Real outputs fed back to LLM — shifts from "reciting knowledge" to "investigating scene".

Main Loop — Boundary between Agent and ChatBot: call model, execute requested tools, feed results back, repeat until done or stuck. Freezes tool set and permissions at start of each round so mid-round config changes don't create mixed-rule execution; changes take effect next round.

Tool Routing — Registers all available tools. Model only says "call X with params Y." Claude Code unifies built-in, MCP, client tools under single addressing; model sees no difference.

Model Routing — Decides which model to call this turn; fallback on overload/unavailability. Claude Agent SDK's model and fallback_model params.

Error Classification — Mapping of failure types to handling: timeout/rate-limit → retry; permission denied → human approval; file corruption → rollback; quota exhausted → wait. Claude Code splits success/failure into two hook events: PostToolUse and PostToolUseFailure.

Sub-Agent Scheduling — Dispatch parallelizable sub-tasks to independent sub-agents, each with clean context; main thread window not saturated; permissions same as caller to prevent privilege escalation. Claude Code's sub-agents.

Most critical module: Main Loop. Two foundational technologies:

ReAct (2022) — alternates Reasoning and Acting: think one step, act one step, observe, think next. This is the conceptual seed of the main loop.

Function Calling (OpenAI, 2023) — lets LLM emit structured "which tool, what params" for stable integration with deterministic code.

This is the Harness's indivisible core; all agents' main loops are continuations and engineering of these two.

4.3 State Domain

Fills "doesn't remember last call". Agent is a stateful app: session, memory, task progress need persistence.

Session Recording — Append-only log of every step in order; survives process crash for resume; enables post-hoc audit. Codex persists as JSONL.

Session Management — Container for one task: create, resume, fork, interrupt. Human closes terminal, continues next day. Claude Agent SDK's resume and fork_session.

Record Retrieval — Session log is sequential; need indexed search (by file path, success/failure, todo-linked history). Used when assembling input and when restoring session to locate breakpoint. Codex uses SQLite for indexing.

Checkpoint — Snapshot before action for full restore on failure — prerequisite for daring to let agent act. Kiro stores checkpoints in a shadow repo for restore without polluting target project's git.

Task Checklist — Long tasks span many rounds/days; checklist tells each round what's done and what remains. Claude Code's todo tool; Kiro's spec task file.

Memory — Cross-session conclusions and user preferences, stored separately from session log. Types: long/short-term, episodic, semantic.

Core module: Session. Anthropic's analogy: shift-work project where each arriving engineer doesn't know what previous shift did. Task module also critical — persist progress state for resume: "which step, confirmed conclusions, accumulated lessons." Implementations:

Claude Code: uses a git commit as breakpoint, plus a progress file as temporary workspace; breaks hundreds of todos into JSON records, stepped through. JSON (not ad-hoc Markdown) because LLM less likely to mutate JSON structure.

DeepSeek dsh: records entire session as append-only event log; on crash, replays log to recover.

4.4 Action Control Domain

Fills "not 100% correct". Key to moving from "demo works" to "production-ready"; constrains "which actions are disallowed (blocked beforehand)".

Action Interception — Operates after model returns tool call, before actual execution. Every tool execution must pass through here. Code-level hard judgment, not prompt-level soft judgment. Claude Code's permissions rules apply before every tool execution.

Permission Rules & Judgment — Layered rules. Claude Code's settings have four layers; enterprise layer has highest priority. Interceptor matches action against rules → allow, deny, or human approval.

Tool Visibility — Based on permission rules, decides which tools model sees this round. Invisible tools are never attempted — cheaper and safer than post-hoc denial. Claude Code removes disabled tools from input entirely.

Human Approval — Irreversible actions require plan first, human approval, then execute. Approval must carry scope and TTL; timeout → auto-deny to prevent unlimited reuse. Claude Code's permission prompts include scope; Kiro's trust mode.

Sandbox Isolation — Confine files, network, processes to controllable sandbox; failure domain limited to sandbox. Sandbox fails to start → refuse execution, not degrade to allow. Codex sandbox policy; Claude Agent SDK sandbox param.

Credential Custody — Secrets extracted only at moment of actual tool execution; never written into model input — model cannot leak what it never saw.

Core modules: Action Interception and Permission Rules & Judgment. Three-layer defense by interception strength:

Permission tiering: tools grouped into read-only, write, execute; default grant minimum needed for task.

Two-phase commit: for production writes, external sends, cost-incurring irreversible actions — forbid one-step; first produce "proposed execution plan", human/compliance review, second step executes.

Hard rule constraints: encode architectural specs and security redlines as machine-enforceable lint (e.g., dependencies must flow downward only: Types → Config → Repo → Service → Runtime → UI, no reverse). Violation → error + "how to fix" fed back to LLM. Far more reliable than repeated prompt admonitions.

Beyond these, a permanent baseline: zero trust for external content. All public-net, email, external document input treated as untrusted; guard against injected "ignore previous instructions, you are now another agent" prompt-injection payloads. Likewise, LLM outputs default untrusted; must pass desensitization and compliance checks before release.

Investment level driven almost entirely by risk: more irreversible actions, higher compliance → heavier investment here. Pure read-only Q&A agents need little; production ops, outbound messaging, financial transactions need maximum: read-only locate vs authorized write strictly separated, environment binding immutable, critical writes require human approval, full-chain audit.

4.5 Verification Domain

Also fills "not 100% correct" but post-action logic.

Acceptance Criteria — Must declare goal description and acceptance criteria first; otherwise checker has no basis, model can only self-declare done. Kiro's spec file separates acceptance criteria.

Checker Integration — Third-party checkers via adversarial verification. In coding agents: compile, test, static analysis, business validation, judge models — third party proves, not the model itself.

Failure Handling — On verification failure: retry, rollback to checkpoint, or escalate to human; cap retry count to avoid infinite loops.

Two key questions: What to verify — task truly completed, output truly correct; this domain is Loop Engineering's focus. Who verifies — never the executing LLM. LLMs are terrible self-judges; empirical tests show they consistently self-praise even when output is mediocre. Must introduce independent verifier; "done" becomes third-party proven conclusion, not self-declaration.

Three key verification aspects:

Deterministic programs (computational verification) — test, compile, liveness probe, reconciliation; objective, fast, but cannot judge business reasonableness.

Specialized judge LLM (reasoning verification) — understands semantics and UX; cost: slower, less stable, extra cost.

Human review when necessary .

Two empirical rules:

Separate "production" and "acceptance" roles; deliberately tune verifier to be more suspicious. Making an independent evaluator picky is far easier than making the generator harsh on itself.

Verify "with environment" — not reading conclusion or glancing at screenshot, but actually walking through like real user/system: open app, call API, check DB state and monitoring.

Failure Handling has three aspects:

Limited self-healing: on step failure, feed error back to LLM for self-correction, but cap retries to avoid spinning.

Crash recovery: after unexpected process termination, replay logs to breakpoint, not restart from scratch.

Cleanup on exit: regardless of outcome, clean temp resources, release locks, explicit "commit or rollback" — no dirty state for next task.

4.6 Access Domain

Access Gateway — Single endpoint for multiple client types; ingest requests, emit responses; swap clients without touching agent core. Trigger sources not limited to humans — alerts, schedules, external events enter here. Unattended sessions: approval timeout must auto-deny. Kiro uses ACP + custom WebSocket for 4 client types; openclaw receives from WhatsApp, Telegram, Slack.

Reverse Request — Channel for agent to proactively contact human, e.g., request authorization. Claude Agent SDK's three reverse requests.

4.7 Policy & Extension Domain

Carries the quantitative conclusion from Chapter 1: same Harness serves different verticals, so anything that varies by vertical must be externalized as declarations, not baked into core code. Inference rule: if changing vertical requires changing it, it belongs here; if it stays same across verticals, it does not.

Layered Config Loading — Supports custom-specified configs: permission rules, extensions, hooks. Kiro's enterprise config can only tighten; validation failure → refuse start.

Extension Loading — Entry point for third-party code: Skills, MCP, Plugins. Claude Code provides Plugin, MCP, Skills loading.

Hook Loading — Delivers events to attached hooks, executes third-party code. Claude Code hooks cover session, prompt, tool call, permission, sub-agent, context compression, etc.

Dynamically loaded content has exactly three categories:

Policies & Rules: change what is allowed — same tools, different permission boundaries.

Extensions: change what can be done — plugins, MCP servers, tool and sub-agent definitions.

Hook Points: change when to interject — insert vertical-specific flow at a lifecycle point.

Policies & Rules carrier is an evaluable match table, not code . Shape: each entry = tool name + parameter pattern, grouped by result (allow/deny/ask). Example JSON:

{
  "permissions": {
    "deny": ["Bash(rm -rf *)", "Read(./.env)"],
    "ask": ["Bash(git push:*)"],
    "allow": ["Bash(git status)", "Read(./src/**)"]
  }
}

Permission Rules & Judgment is a pure function: input = this action + this table → output = allow/deny/escalate; deny wins. No line in core says "must not drop database"; that sentence lives entirely in data — swap table, swap permission boundary.

Declaration unit can be raised one level. Kiro matches on capability tags (e.g., file-write, command-exec) not tool names; tools carry tags. New tool just gets tag → automatically covered by existing rules without touching rule file. If declaration unit is tool name, every new tool needs new rule, and prior rules don't apply to it.

Three loading times; only third is truly runtime-dynamic:

Process start: load by layer and merge; enterprise deny items cannot be overridden by lower layers.

Round start: re-read and freeze for this round; config file edit takes effect next round without restart.

Runtime append: on approval, append a temporary rule with scope and TTL (e.g., allow push for this session). This is the line between Human Approval and Permission Rules & Judgment in the diagram.

Final discipline: parse failure → always treat as "deny". Most tempting to do opposite (ignore and continue) because it's easiest, but in policy context that semantics means removing all restrictions.

4.8 Observation Domain

Audit Recording — Write-only log of which rule each judgment relied on, approval scope; post-hoc answer to "why was it allowed then?" Codex writes judgment basis into same event stream as conversation.

Usage Metering — Write-only log of tokens, latency, cost; billing basis and fed back to Context Window Quota for budgeting. Claude Code's cost/usage stats.

Observation Data Export — Push records in standard format to external logging, monitoring, audit systems; export success rate, step count, cost, human intervention count for offline evaluation. Otherwise data stuck on local machine. Claude Code exports metrics/events via OpenTelemetry.

Process Replay — Reconstruct record into human-readable process; debugging and retrospective can step through what happened, not just see final result. Claude Code can restore historical sessions and replay step by step.

Evaluation — Score a batch of runs by unified metrics: success rate, avg steps, per-task cost, human intervention rate — so a change's impact is measured, not felt.

Optimization — Take evaluation results and tune adjustable parts: prompts and project rules, context budget and compression triggers, tool descriptions, model selection and retry counts — close the improvement loop.

Six modules split into two groups. First four only accumulate and present records: audit, metering, export, replay — all write-only. Reason: debugging, audit, evaluation have different purposes but need same records (what rule each judgment used, what each call cost). Debugging asks "why just now?", audit asks "why allowed then?", evaluation asks "which part most expensive/slow?". Last two consume records: evaluation scores by unified metrics, optimization feeds scores back to tune knobs.

Write-only is a hard constraint, not a suggestion. If any domain reads observation records for judgment, observation becomes a control dependency; traceability loses credibility because the audited starts depending on audit records. This constraint targets in-loop judgment; evaluation and optimization read out-of-loop, not participating in any action allow/deny decision, so no control dependency.

Note: evaluation and optimization only get interfaces and contracts in this domain, because metric definitions and optimization directions are entirely scenario-dependent; general architecture cannot supply implementations. Planner remains outside Harness — that boundary unchanged.

5. Software Architecture Diagram

Figure 7 shows runtime layout of eight domains. Not all 36 modules needed in every scenario. Coding agent vs production ops agent invest vastly differently in Verification and Action Control. Next chapter introduces five-dimensional coordinates to turn "which to build, how thick" into a scorable decision.

6. Applying Framework to Vertical Scenarios

6.1 Task Scenario Five-Dimensional Coordinates

Five dimensions suffice to judge a vertical's implementation center of gravity:

Verifiability: Can correctness be judged low-cost, objectively, fast? Coding: very high (compiler, tests = honest judges). Fault diagnosis conclusion quality, weekly report quality: hard for machine to judge on the spot.

Reversibility: Can a wrong action be undone/rolled back? Code change: reversible (git revert, sandbox reset). Restart production cluster, delete data, send external notification: mostly irreversible.

Environment Determinism: Is runtime stable and reproducible? Same code in sandbox today = tomorrow. Production: dozens of clusters, multiple data centers; same command safe now, fatal next moment.

Autonomy: One-shot Q&A or continuous multi-hour, multi-session long-horizon task? Longer horizon → higher demand for remembering progress, not losing goal.

Compliance: Regulated? Audit required? Liability for results? Coding on experimental branch: near-zero compliance pressure. Finance, medical, production ops: every action may need trace, review, root-cause traceability.

Mapping each dimension to engineering focus when strict:

Verifiability — Core Question: Can right/wrong be judged low-cost, objectively? Engineering Focus When Strict: Low verifiability: shift from auto-verification to independent review + human gating

Reversibility — Core Question: Can mistakes be undone/rolled back? Engineering Focus When Strict: Low reversibility: pre-action constraints, permission tiering, two-phase approval

Environment Determinism — Core Question: Is world stable and reproducible? Engineering Focus When Strict: Low determinism: strengthen exception handling, repeatedly verify in real environment

Autonomy — Core Question: Single-turn or long-horizon? Engineering Focus When Strict: High autonomy: state persistence, memory, checkpoint resume

Compliance — Core Question: Regulated, audit needed? Engineering Focus When Strict: High compliance: action control, audit, data isolation, traceability

Verifiability can be split further per 4.5's two expectation types, because enumerable vs non-enumerable expectations have vastly different verification cost. Coding is typical: compile/test for enumerable expectations ≈ free; but "this refactor broke no existing invariants" remains expensive. Without splitting, coding agent's verifiability recorded as high, masking its truly expensive half.

6.2 Five Dimensions Directly Determine Which Modules Thicken

This section is the chapter's landing point: translate each dimension's score directly into module checklist.

Low Verifiability — All three Verification modules required; Checker Integration must connect human review channel; Failure Handling must support escalation to human

Low Reversibility — Action Control's Human Approval becomes two-phase commit with scope+TTL; Permission Rules enable non-overridable keys; Checkpoint module atrophies here, so judgment must shift earlier

Low Environment Determinism — Context's User Reference Parsing must ingest real-time state; Action's Error Classification must be fine-grained; Verification must walk through real environment

High Autonomy — State's Session Recording, Record Retrieval, Task Checklist, Memory all required; Context's Compression must support long runs

High Compliance — Observation's Audit Recording granularity increased; Action Control's Human Approval and Permission Rules thickened; unattended sessions must auto-deny on approval timeout

Emphasis: five-dimensional scoring only changes module thickness and selection; does not alter Chapter 4's eight domains and 36 modules structure. Five dimensions don't change architecture, only investment allocation.

6.3 Coding Agent and Production Ops Agent at Opposite Corners

Harness engineering was born, evolved, and matured in coding scenarios.

Claude Code: started as Anthropic internal CLI coding tool; spread virally among engineers; publicly released early 2025. Through repeated polishing and long-horizon coding practice, Anthropic realized: what determines usability is often not the model but the engineering system around it — "loop + tools + context management + guardrails" — and formally named it harness in engineering blog.

OpenAI Codex: pushed harness engineering to large real codebases. To stabilize agent collaboration in huge repos, Codex team codified senior engineers' architectural judgments into machine-enforceable Lint rules, and used a concise AGENTS.md as "navigation map" — classic harness techniques.

DeepSeek dsh: open-source Agent Harness runtime framework. Makes context, tools, orchestration, security all pluggable plugins, with event sourcing as foundation — represents fully framework-izing harness itself.

Different origins, divergent paths, yet all point to same fact: virtually all mature Harness concepts and practices were first honed on coding agents. Hence these experiences are strongly tied to coding scenario, and conversely coding is the most Harness-friendly scenario because it enjoys three unique advantages:

Low-cost verification: code correctness needs no subjective judgment; compile, run tests, run lint → machine gives objective verdict instantly.

Reversible actions: wrong code → git revert or sandbox reset → world returns to pre-error state.

Deterministic, reproducible environment: same code, same deps → today's run ≈ tomorrow's run.

These three conditions let coding agents adopt "act first, verify later": modify boldly; if error, post-verification intercepts, then rollback and retry. In other words, coding scenario allows engineering focus on post-action verification, while pre-action constraints can be relatively relaxed.

Problem: vertical production scenarios often have the exact opposite conditions: verification not free, actions hard to roll back, environment not deterministic. Taking a harness "tuned for friendliest scenario" and applying it directly to a scenario with completely opposite fundamentals almost guarantees failure.

Comparison across five dimensions:

Verifiability: Coding Agent: High (compile, test, pipeline); Production Ops Agent: Medium (some metrics queryable but conclusions often need human judgment)

Reversibility: Coding Agent: High (revert commit or reset sandbox); Production Ops Agent: Very low (restart, scale, config changes mostly irreversible)

Environment Determinism: Coding Agent: High (reproducible); Production Ops Agent: Low (multi-cluster, multi-DC, constantly changing)

Autonomy: Coding Agent: High (continuous autonomous delivery for hours); Production Ops Agent: Constrained (critical actions require human-in-the-loop)

Compliance: Coding Agent: Low; Production Ops Agent: Very high (operation logging, audit, accountability)

They are nearly opposite on all five dimensions; therefore harness designed for coding cannot be copy-pasted to vertical scenarios. This also explains the five gaps in Chapter 4's table: the four coding agents score high on verifiability and reversibility, so they don't need Verification domain or Credential Custody module, and leave Evaluation & Optimization to offline human judgment.

Verifiable & reversible scenarios (e.g., coding): focus on post-verification; pre-constraints light.

Low verifiability, irreversible, high compliance (e.g., production ops, finance, office): focus must shift to pre-action — block risk before action occurs — and make post-action audit and traceability rock-solid.

Closing analogy: Coding agent is like working on scratch paper — write wrong, cross out, rewrite; cost is a few sheets, so encourage many attempts, gate at the end. Production ops agent is like performing surgery — every cut acts on real patient and cannot be undone; real skill lies entirely before the cut: repeated confirmation, tiered authorization, critical steps require co-sign; gate at the start of every action, and leave complete surgical record for traceability afterward.