Mastering Agent Harness: The Core Architecture Behind Modern AI Systems

Why a Harness Is Required

Large language models are next‑token predictors that lack persistent context and are stateless. A Harness supplies a structured, auditable layer that preprocesses user input, builds rich prompts, routes tool results back into the dialogue, manages memory, and enforces guardrails for safe output. Martin Fowler describes the Harness as a governance mechanism that shifts engineering focus from writing isolated code to constructing systems, scaffolding, and feedback loops that enforce invariants before deviations propagate.

Harness Use in Long‑Running Conversations

Context is a scarce resource; overloading the model with the full transcript degrades signal quality. A well‑designed Harness curates the context presented at each step, separating a high‑relevance working memory from a full transcript that is displayed selectively. The Harness implements a bidirectional control structure: a feed‑forward side (architecture docs, rules, execution plans) and a feedback side (linters, test runners, code‑review agents, structural analysis tools) that together form an iterative loop reducing failure modes.

From Language Model to Harness

Four concepts stack vertically:

Language Model : raw next‑token predictor with no memory or tools.

Reasoning Model : the same model augmented by prompts or fine‑tuning to perform intermediate reasoning, validation, and self‑check.

Agent : adds a control loop that decides which tools to invoke, updates shared state, and determines task completion.

Harness : the outer software layer that governs context, tool exposure, prompt construction, state tracking, layered memory, and overall control flow. Fowler likens the Harness to a governance mechanism that steers the Agent from an initial state to a desired outcome while preventing technical debt and architectural drift.

Core Components of an Agent Harness

Live Repository Context

Agents must see the relevant parts of a codebase—files, functions, module boundaries, and dependency graphs—dynamically as they browse, not just a static snapshot.

Prompt Shape and Caching

Each model call incurs latency and cost. Mature Harnesses cache stable prompt prefixes and deliberately allocate space for system instructions, tool results, and dialogue history, avoiding reconstruction of the entire context on every call.

Tool‑Use Guidelines and Boundaries

Harnesses define when a tool may be invoked, how results are formatted, and guardrails that prevent misuse, ensuring the Agent operates within predictable, invariant structures.

Context Bloat Management

When conversations grow, Harnesses compress, summarize, or discard irrelevant content to keep the context window within limits. This discipline distinguishes professional Harness engineering from ad‑hoc implementations.

Layered Memory Architecture

Memory is split into a working memory holding distilled high‑priority state and an archival transcript storing all content but presented selectively. The Harness decides what moves between layers and when boundaries shift.

Delegation via Subagents

Complex tasks are broken into sub‑agents (e.g., code‑review agent, document‑search agent, test‑execution agent) that report back to a coordinating agent, keeping each sub‑agent’s context manageable while enabling the system to tackle large problems.

Example Prototype: Essay Harness

The prototype demonstrates the principles by treating writing as a four‑stage pipeline. A single LLM call cannot reliably plan, retrieve, synthesize, and draft simultaneously; attempting to do so produces hallucinations and off‑topic output.

Four‑Stage Pipeline

Stage 1 – Planner runs at temperature 0.2, receives the current memory state, structured source summaries, dialogue transcript, and draft, and returns a JSON plan specifying the current phase, reasoning mode, draft goals, required source questions, style directives, and tool calls with parameters. If a single source is present but the plan lacks a read_book call, the Harness injects it automatically. On LLM failure, the Harness falls back to buildDefaultPlan(), which infers intent from user keywords.

Stage 2 – Tool Execution is deterministic and involves five tools: search_library: BM25‑style term overlap search with normalization, stop‑word removal, and scoring by exact, partial, and frequency matches. read_book: deep reading of a single source, scoring each note and idea against the current query and returning the strongest excerpts with full chapter indices. compare_books: extracts feature vocabularies from each source, identifies shared concepts, and generates a structured intersection and tension report. read_document: extracts relevance‑scored excerpts from uploaded PDF or DOCX files. inspect_wiki: searches internal reference pages.

Stage 3 – Evidence Synthesis runs at temperature 0.2. It extracts a compact evidence package containing:

Best working argument.

Narrative arc supporting the argument.

Four to six outline points.

Three to six logical steps forming a proof graph.

Strongest cross‑source intersections and identified evidence gaps.

Stage 4 – Drafting receives topic, audience, tone, plan, working memory, evidence package, tool trace, and current draft, operating at temperature 0.35. Its system prompt enforces style, requires evidence‑backed claims, switches between single‑source and cross‑book synthesis modes, and forbids fabricated citations.

Session Architecture and Persistent State

Each writing task instantiates a session persisted as a JSON file on disk. The session records:

Agenda, audience, and tone.

Source list.

User messages, tool results, assistant replies.

Markdown draft.

Pending proposals awaiting approval.

This design ensures auditability, reproducibility, and human oversight.

Work Memory as Structured Knowledge

The work memory object stores not only dialogue but a structured understanding:

Current argument.

Narrative arc.

Dynamic outline.

Proof graph.

Source intersections and tensions.

Evidence gaps and open questions.

Persistent style directives.

Source ledger and recent retrievals.

Pipeline suggestions.

The function mergeMemory() deduplicates arrays, appends new findings, and preserves existing content unless explicitly overwritten, allowing knowledge to accumulate across rounds.

Proposal and Approval Loop

After drafting, the Harness presents one to three conservative draft updates as structured cards showing rationale, change summary, and a diff against the existing text. The user may accept, reject, or provide feedback, preserving human agency and aligning with the principle that the human guides rather than merely observes.

Constrained Context Management

To fit within limited local model windows, the Harness caps:

Transcript length to 30 rounds.

User messages to 900 characters.

Assistant messages to 700 characters.

Tool results to 1000 characters.

The function summarizeContextForPrompt() creates a structured summary containing source metadata, the top‑5 relevant notes, the top‑5 ideas, and a chapter index. Full content is loaded only when explicitly requested, implementing a delayed‑load strategy.

LLM Routing and Graceful Degradation

All three pipeline stages are routed through a single generateJson() abstraction, decoupling pipeline logic from the backend (local Ollama or OpenAI). If Ollama is unavailable, the Harness attempts to start ollama serve and waits up to 12 seconds before falling back to OpenAI, recording the fallback for audit. Each stage has a graceful fallback (planner → buildDefaultPlan(), synthesizer → existing work memory, drafter → evidence‑based scaffold) so the system never returns an empty response.

Significance of the Prototype

The Essay Harness validates the theoretical framework:

Separation of concerns via a layered pipeline.

Structured, cumulative memory.

Deterministic tool execution.

Feed‑forward (architecture docs, rules) and feedback (linters, test runners) control loops.

Human‑in‑the‑loop approval before any state change.

Hallucinations are mitigated not by better prompts but by architectural enforcement that requires evidence before any claim. Context management emerges as a design discipline rather than a tunable parameter, with delayed loading, structured summaries, memory merging, and evidence‑package compression built into the system from the start.