From LLM Call to Full Harness: Understanding Agent Runtime Layers

Complexity Is Built Step by Step

The simplest system is a single line: answer = LLM(prompt). The model reads input, generates text, and ends. It does not remember previous turns nor know if its commands executed.

For multi-turn dialogue, the application must reassemble context before each call:

context = [system_rules, chat_history, user_input]
answer = LLM(context)

This does not give the model a memory block; the application feeds relevant history each time to create continuity.

Context assembly is the first runtime duty of an Agent system. Which rules, files, and history enter the window directly shapes what the model can see next.

To fix an interface timeout, the model must read logs, open code, modify files, run tests, and decide next steps from results. A loop wraps the call:

while not finished:
    response = model(context)
    if response.has_tool_call:
        result = tool_router.execute(response.tool_call)
        context.append(response, result)
    else:
        return response

This is the basic ReAct form: model proposes action, environment returns observation, observation feeds next decision. The model still only generates tokens; the loop drives continuation.

Tools need declared schemas, call IDs, parameter validation, and result correlation. The Tool Router translates model intent into deterministic calls; Tool Results return to context. Files, terminals, browsers, and external services extend this action protocol.

As tasks grow, gaps appear: context windows cannot hold full history (requiring working sets, summaries, persistent sessions); tools modify files and access networks (needing permissions and sandboxes beyond prompts); tasks may be cancelled, retried, or continued on another client (requiring events, checkpoints, recovery). These capabilities gather around the Agent Loop, forming the Harness.

Harness places the model's proposed next step into explicit context, permissions, state, and verification rules.

Where Each Layer Stops

Mapping this evolution to the "fix interface timeout" task clarifies boundaries:

Model only proposes the next step. It may request file reads, API calls, or commands, but generating a tool call does not actually change files or databases.

Loop turns intent into action, receives tool results, and decides whether to continue. It cares how actions chain, but cannot alone decide which directories, credentials, or networks a call may touch.

Harness takes over runtime boundaries: it prepares context, constrains tools, saves state, handles cancellation and recovery, and uses tests, metrics, or human approval to judge completion.

With this split, failures land in specific places: context may miss constraints, tool data may be unparsable, state may not persist, verification may check only syntax not results.

Figure 1: Model proposes the next step; Harness puts that step into a controllable runtime.

What Path a Task Follows

Returning to the timeout fix, the runtime roughly follows this flow:

Figure 2: On verification failure, the system continues, retries, or escalates from checkpoints rather than restarting unconditionally.

A key distinction: context is for the model; events are for the system. The model only needs content relevant to the current decision, but the system must retain the full process to answer "what executed", "which step changed a file", "which version the test ran against". Hence full history, current working set, and runtime operations are stored separately.

Pi's Session exemplifies this separation. Sessions are recorded as entries with parent-child relationships; current context is projected from a branch. When nearing limits, the system compresses the working set while preserving original records. Compression changes what the model sees next; facts already occurred remain. This is more controllable than repeatedly stuffing entire chat history into the model.

Loop Is More Than "Call Model Again"

Compressing the above chain yields a minimal Agent Loop:

Think   Decide next step from goal and context
Act     Call tool or execute action
Observe Read action's returned result
Verify  Judge if result meets current phase condition
Repeat  Continue, stop, or escalate to human

The ReAct paper describes the basic interleaving of reasoning and acting. To fix a failing test, an agent can read logs, hypothesize cause, open relevant files, modify code, run tests, then decide to continue or stop based on results.

ReAct solves "how reasoning and action interleave"; Verify, stop conditions, and human takeover are the engineering additions for a production Loop.

Production must also ask: what state did this step change? Can failure be retried? What are time and budget limits? When the same error repeats, should the model keep guessing or hand off to a human?

Production must add verification, stop, state, recovery, isolation, and observability.

Without stop conditions, an agent may retry the same failure endlessly.

Without state, recovery can only re-guess.

Without isolation, two tasks overwrite each other's workspace.

Without observability, occasional success is hard to reproduce.

Stop conditions fall into three categories:

Goal stop : delivery conditions met (tests pass, change generated, approval done).

Resource stop : time, tokens, tool calls, concurrency, or cost hit limits.

Risk stop : privilege escalation, data scope change, repeated failures, or irreversible side effects require human intervention.

Chat can continue in uncertainty; a running Agent must hand uncertainty explicitly to the system — this is the core difference.

Context Needs Right Amount and Explainability

Andrej Karpathy (2025) used "context engineering" to distinguish it from narrow "prompt engineering". Production systems must organize task instructions, examples, retrieved content, tools, state, history, and compression simultaneously.

Too little information starves the model; too much raises cost and distracts judgment.

Inside Harness, the difference lies in how context is assembled.

Early Cursor fed lint and type errors back into context after each edit, and limited per-turn read range and tool calls. As models learned to discover information themselves, Cursor reduced static prompts and increased dynamic context, letting the model fetch history, terminal state, or relevant tools on demand.

The runtime first decides what the next step needs, then assembles context on demand. The assembly process grows complex: which references were taken, which helped, whether context changes improved results — all must be logged.

OpenAI's Codex engineering practices treat the repository knowledge base as system record, a short AGENTS.md as table of contents, then progressively unfold information via docs, execution plans, structural constraints, and check tools. Benefits: short entry point, detailed knowledge loaded per task, rules versioned with code, stale content can be detected.

In the "fix interface timeout" task, four content types should not be mixed:

Task context : goal, acceptance criteria, time and budget.

Repository context : module boundaries, coding rules, relevant tests, change process.

Runtime context : current branch, process state, tool results, completed actions.

Decision context : rationale visible to system when model picks a file, parameter, or next verification.

The first two are relatively stable knowledge; the latter two change during execution. Stuffing all into one long prompt makes maintenance hard and obscures whether an error came from missing rules, stale state, or model judgment.

Context engineering must answer three questions: what did the model see, why did it see that at this moment, did the next step become more reliable? Without recording all three, prompt tuning becomes guesswork.

Tool Calls Turn Reasoning into Side Effects

When a model outputs text, errors stay in the answer. When it calls write_file, bash, or a payment API, errors enter the real system.

Tools cannot be mere "functions the model can call"; they need execution boundaries. Four concerns must be separated:

What the model requested, and whether parameters match schema.

Whether Harness allows this call, and whether human confirmation is needed.

In which process, directory, network, and credential scope the tool executes.

Whether execution succeeded, and whether it can be safely retried.

Pi's default core provides only read, write, edit, and bash; other capabilities enter via extensions, Skills, and packages on demand. This reduces per-request tool description and context cost.

Security boundaries are explicit: extensions and built-in tools run within Pi's existing process permissions. Hooks can confirm and audit but cannot replace process sandboxes. Untrusted code, sensitive credentials, and network limits still require an environment that can restrict files, processes, network, and credentials.

Pre-tool confirmation answers "is this allowed?"; sandbox answers "even if allowed, what is the maximum impact?" Putting both in the model prompt makes post-mortem tracing hard and cannot prove an action was actually constrained.

Tool results also need verification. Structured calls let Harness check fields and types, but schema correctness ≠ business correctness. To replay a call, at minimum you need state, version, source, error type, and retryability.

After writing a file, check diff; after changing an interface, run tests; after calling an external service, confirm return status and idempotency key. The model can propose "run tests first"; the verifier then judges if test results satisfy delivery conditions.

Tool call protocols only constrain parameter shape; they cannot replace security boundaries. JSON Schema cannot limit which directories a command touches; "confirmation required" cannot substitute process-level network isolation. An API returning "request received" does not guarantee the side effect committed; final state must be verified against the interface contract.

Safety, idempotency, and rollback must be jointly borne by runtime and backend.

Long Tasks Expose State Problems First

Short conversations can keep state in memory; long tasks quickly reveal issues.

After tens of minutes, at least three timescales appear: user-visible conversation time, actual tool execution time, and model's next decision time. Parallelism, retries, cancellation, and recovery desynchronize them.

If the system only saves the last message, recovery cannot know whether a command already executed, whether re-execution causes side effects, or whether the current summary is based on stale code.

Session, Operation, and working set must divide labor. Session records what happened; Operation records how a run started, retried, and ended; working set supplies only what the model needs next.

Pi's session tree, compression logs, and branch summaries address this. Codex's Thread, Turn, Item hierarchy similarly separates task, turn, and concrete action.

Recovery needs explicit checkpoints. Re-invoking the model is not a checkpoint.

Which tool results are committed, which actions are replayable, which require human confirmation, whether failure rolls back to before model decision or before tool execution — all must be recorded upfront. Without checkpoints, recovery is just guessing again.

Long tasks also need phase contracts. "Fix timeout" can split into investigation, modification, verification, handoff. Each phase defines inputs, allowed actions, completion criteria, and handoff artifacts, narrowing scope.

Planner decomposes goals, Generator performs local edits, Evaluator judges phase satisfaction. Phases hand off a traceable result, not just chat history.

Sprint contracts don't turn the Agent into a fixed script; they narrow the uncertainty range per round.

Investigation phase can keep gathering info if no reproducible evidence exists; modification phase produces a reviewable diff before verification; verification phase records test version and environment so handoff has evidence.

Sub-agents involve state design, not just parallelism. Cursor's practice gives sub-agents fresh context windows to avoid contaminating the main session with model switches or extra tasks.

After context isolation, the main Agent must still know sub-task inputs, outputs, permissions, and lifecycle; otherwise you just have more invisible state.

Verification Loop Determines Whether the System Improves

Models generate fast, but system delivery isn't necessarily fast. Production quality depends on how short and clear the post-generation verification loop is.

Cursor's Harness article examines offline benchmarks and online signals together. Public benchmarks compare framework changes; real-world continuation actions, error rates, and user retention of edits provide another signal set.

Tool call reliability must be monitored separately. A single tool failure can make subsequent reasoning build on wrong state.

OpenAI's Harness engineering exposes logs, metrics, traces, runnable worktrees, and browser actions to the Agent, letting it reproduce issues, verify fixes, and write results into the change process.

The value: feedback previously visible only to humans becomes readable and executable by the Agent.

Verification splits into three layers:

1. Action verification : parameters, permissions, process, and resource scope compliance.

2. Result verification : tests, metrics, state, and output meet task conditions.

3. Process verification : context, tools, retries, and handoffs leave traceable records.

Only layer one yields safety but possibly wrong results; only layer two yields occasional success but hard to reproduce; missing all three makes the system harder to maintain as model capability grows.

Verification need not wait until the end. Investigation uses reproduction scripts to confirm the problem; modification uses static checks and diffs to limit scope; verification runs full tests and integration checks. The earlier feedback re-enters the Loop, the less likely the model continues reasoning on erroneous state.

Harness Must Also Learn Subtraction

Early Harnesses often compensate for model weaknesses: preload more instructions, filter tool results for the model, force reset on context limits, add orchestration for unstable actions. These were necessary guardrails at the time.

Anthropic later reminded: when model capabilities change, re-check which control logic still works.

Resets that prevented premature stops may become extra interruptions on new models; filters that reduced context noise may block the model's own information discovery paths.

Models can take more local decisions, but safety, cost, approval, caching, audit, rollback, and UX remain Harness responsibilities. Regular inventory: which local orchestration, context preloading, and memory selection now make decisions the model could make itself.

On model upgrades, run a set of real tasks comparing old Harness, new Harness, and bare model trajectories; observe success rate, tool errors, context consumption, recovery count, and user takeover points. Keep logic that still shows benefit; delete or downgrade logic that lost benefit and adds complexity.

Harness evolution must do two things simultaneously: make permissions, state, verification, and observability reliable, and remove intermediate layers that no longer carry weight.

Stacking features makes the system heavy; cutting boundaries too fast pushes risk back to the model. Trade-offs need evidence.

Where Different Harness Trade-offs Diverge

Looking at Pi, Cursor, and Codex together, there is no single "standard Harness". The difference lies in how deep the control plane goes.

The comparison below summarizes responsibilities and trade-offs from public practices, not a full product evaluation.

Pi keeps core thin: few default tools, explicit resource loading, extensible TypeScript interfaces, session compression and branching. Suits teams wanting full workflow control; trade-off: permissions, sandbox, sub-agents must be supplied by extensions or external environment.

Cursor invests in context discovery, model adaptation, online evaluation, and product feedback. Fits continuously iterating Agent products; trade-off: runtime must maintain more complex dynamic policies and experimentation infrastructure.

Codex goes heavier: threads, turns, actions, approvals, sandbox, repo knowledge, and structural checks integrated into a sustainable engineering process. Suits long tasks and high audit requirements; trade-off: infrastructure, rule maintenance, and state management are more complex.

Figure 3: Each approach has its applicable range; differences lie in control plane depth and operational cost.

Task risk, recovery requirements, number of models and clients, and the team's willingness to bear operational cost decide which responsibilities go into Harness core, which stay in project files, extensions, or backend platforms.

Harness Is Approaching the Control Plane

Traditionally backend and Agent Harness were separate: backend provided APIs, queues, databases, logs; Harness handled model loop and tools. With many long tasks, state, permissions, and traces scattered across both sides.

Agents compose capabilities at runtime. Backend must also provide capability discovery, state reads, durable execution, permission boundaries, and unified tracing.

Viewed this way, in long-task systems Harness can be understood as the control plane between model and backend:

It governs what the model can do next, what actually happened, and how to recover when things go wrong; the backend is the execution plane it observes and mutates.

This control plane must answer four questions: current task state, what can be called next, how to recover from action failure, who confirms completion.

It need not be a standalone service; it can be composed of runtime, repo rules, and backend protocols, but responsibilities must have clear ownership.

This also explains the position of semantic decision layers like Jev. Judgments with limited answer spaces can be extracted from general generation calls into decision modules better suited for audit and rollback.

Harness then connects those judgments to permissions, tools, and state. Model, decision engine, backend each handle their strengths, making failures easier to locate at specific boundaries.

In practice, I check four questions first:

After task failure, can we identify the last definitively completed action?

Can the context the model sees next be explained by source and trimming rationale?

Does every tool's side effect have permission, sandbox, and idempotency policy?

Are verification results and runtime events readable by both humans and Agents?

If these questions lack answers, adding more tools and stronger models often only widens the debugging scope.

My usual trade-off: first complete the control plane, then see if model capability translates into stable system behavior.

After Agents enter production, the costliest part is often failure handling: can a failure be seen, explained, recovered?

Harness manages the distance between "what the model said" and "what actually happened in the system."

References

Cursor: "Continuously Improving Our Agent Harness" (https://cursor.com/cn/blog/continually-improving-agent-harness)

OpenAI: "Harness engineering: leveraging Codex in an agent-first world" (https://openai.com/index/harness-engineering/)

Anthropic: "Harnessing Claude's Intelligence" (https://claude.com/blog/harnessing-claudes-intelligence)

Anthropic: "Harness design for long-running application development" (https://www.anthropic.com/engineering/harness-design-long-running-apps)

ReAct paper: "ReAct: Synergizing Reasoning and Acting in Language Models" (https://arxiv.org/abs/2210.03629)

Pi official repo (https://github.com/earendil-works/pi)

Mario Zechner: "What I learned building an opinionated and minimal coding agent" (https://mariozechner.at/posts/2025-11-30-pi-coding-agent/)

Andrej Karpathy, Context engineering (https://x.com/karpathy/status/1937902205765607626)

Simon Willison: "Context engineering" (https://simonwillison.net/2025/jun/27/context-engineering/)

"Loop Explained: From ReAct to Loop Engineering, What Does Agent Loop?" (2026-06-20)

"Key Changes in Agent Architecture: Harness Becoming the New Backend" (2026-05-02)

"From Pi: Agent Harness — Doing Less for More Controllable Runtime" (2026-09-08)