Code Harness vs. Model-Driven Harness: Can Agent Control Be Expressed as Executable Natural Language?

"If you're not the Model, you're the Harness."

The quote, originally from LangChain, highlights the importance of the harness layer in building LLM agents. The article first defines three related engineering concepts: glue code (connector code without business meaning), middleware (infrastructure services such as message queues, RPC frameworks, API gateways), and harness (runtime control logic around an agent that decides when to call the model, how many times, validation, retries, and multi‑step orchestration). Although all three belong to engineering, their problems and paradigms differ, and harness logic is often tightly coupled with controller code, making it hard to inspect, transplant, or compare.

01 Control Strategies for a Single Agent Run

Code harness: hard external control via program logic.

Natural‑Language Agent Harnesses (NLAH) + Intelligent Harness Runtime (IHR): move the control strategy into a readable natural‑language document that a shared runtime executes.

Self‑harness: a future design where a controller model directly drives other models without any external harness layer.

Modern LLM agents are multi‑step systems that use tools, maintain state, recover from failures, and sometimes delegate to sub‑agents. An external harness layer organizes these behaviors and significantly impacts measured performance.

02 NLAH and IHR Architecture

The paper " Natural-Language Agent Harnesses " investigates whether the reusable design pattern of an agent harness can be represented as an executable natural‑language object, turning accidental glue code into a scientific representation. It introduces two core artifacts:

NLAH (Natural‑Language Agent Harness): an editable document describing runtime harness strategy—stages, roles, state rules, validation, recovery, stop conditions, etc.

IHR (Intelligent Harness Runtime): a shared in‑loop runtime that interprets NLAH documents and materializes them as audited agent calls, handoffs, state updates, validation gates, and product contracts.

The architecture consists of four layers (illustrated in Figure 1): a minimal base agent (LLM + single tool), a runtime policy expressed as a fixed NL instruction, the NLAH document that encodes the strategy, and scripts/adapters that implement deterministic mechanisms such as JSON‑schema validation, diff computation, format conversion, and external API calls. The analogy compares the base agent to a horse, the runtime policy to a rider, the NLAH document to a route map, and the scripts/adapters to traffic signals and checkpoints.

03 Evaluating the New Paradigm

With the core architecture in place, the authors design three research questions (RQs) and evaluate on three benchmarks: SWE‑bench Verified, Terminal‑Bench 2.0, and OSWorld.

RQ1 (Harness Implementation) compares three implementations of the same harness idea: code harness, prompted NLAH (plain prompt to Codex CLI), and IHR‑executed NLAH. Results show that IHR‑executed NLAH achieves performance comparable to the code harness while dramatically compressing the static strategy (e.g., Live‑SWE reduced from 60.1 k tokens to 2.9 k tokens). The trade‑off is higher token/call overhead in the current prototype.

RQ2 (Mechanism Fidelity) annotates a single NLAH with eight observable mechanisms (contract, tool gate, stage handoff, stop condition, etc.) and measures compliance rates when IHR runs the benchmarks. Contract‑type mechanisms exhibit high compliance because their boundaries are clear; cross‑stage and cross‑sub‑agent mechanisms show significantly lower compliance.

RQ3 (Module Ablation) performs leave‑one‑out ablations by disabling one NLAH module at a time (file‑backed state, self‑evolution, multi‑candidate search, context compression, markdown memory, etc.) without changing any underlying code. Findings: file‑backed state yields the largest positive gain (+2.6 % / +13.9 % success); self‑evolution gives the highest absolute performance but incurs large token cost; multi‑candidate search and context compression hurt performance; markdown memory’s impact varies by task. The conclusion is that more harness modules are not always better—effective practice should retain externalizable state, use self‑evolution cautiously, and prune harmful modules.

04 Limitations and Risks

The authors note two main concerns. First, natural language is imprecise: important constraints may be under‑specified, interpreted differently by models, or weakened by edits. Therefore, precise mechanisms remain in code, and runtime behavior must be verified empirically rather than inferred from text. Second, externalizing harness logic lowers development cost and improves comparability, but it also lowers the barrier for malicious workflows. Since the harness mediates tool usage, artifact handling, and delegation, it can become an attack surface for prompt injection, malicious tool grafting, or supply‑chain contamination. Deployments should therefore incorporate provenance tracking, auditing, permission controls, and sandbox isolation.