How Anthropic Advances Agent Development: From Code Writing to 4‑6 Hour Autonomy

Problem Shift: From Code Generation to Long‑Running Autonomy

While many discussions still ask whether large language models can write code, Anthropic’s article Harness design for long‑running application development focuses on the next practical challenge: once a model can code, how to orchestrate it so it can plan, implement, evaluate, and deliver a usable product over several hours.

Key Insight – Separate Generation and Evaluation

Anthropic found that the decisive factor is moving from model capability to harness design. They split the system into three explicit roles:

Planner : expands a short requirement into a full product specification.

Generator : writes code and implements features.

Evaluator : runs the product, finds bugs, gives feedback, and forces rework.

The evaluator is a separate agent that actually executes the UI, calls APIs, and checks a database, providing an external friction layer that prevents the model from prematurely declaring a task complete.

Self‑Evaluation Problems

Anthropic notes that LLM self‑evaluation is unreliable: the model is lenient, treats half‑finished work as finished, and prefers “looks good enough” over truly functional results. This manifests in both design tasks (no visual identity, no design decisions) and engineering tasks (missing critical defects, treating stubs as features, believing the final mile is done).

Context Anxiety and Reset Strategy

When a long task fills the context window, the model exhibits “context anxiety” – it starts wrapping up even if work remains. Anthropic’s response is to reset the context by spawning a new agent and handing over the state via a structured artifact, rather than merely compressing history (compaction). Reset avoids lingering anxiety but increases orchestration complexity, token cost, and latency.

Front‑End Design Example

Using a front‑end design prompt, Anthropic showed that a solo run produces a clean but generic page, while a harnessed run iterates through multiple feedback rounds, eventually producing a museum‑quality design with 3D perspective, chessboard floor, and room‑through navigation. The evaluator’s scoring criteria (overall design, originality, craftsmanship, functionality) steered the model toward more expressive results.

Full Harness vs. Solo for a 2D Game Maker

They compared a solo run (20 min, ~$9) with a full harness run (6 h, ~$200) for building a retro game editor. The solo version suffered layout waste, broken workflow, and an unplayable game (missing entity‑runtime connections). The full harness version expanded the feature set to 16 items across 10 sprints, added sprite animation, AI‑assisted generators, sound, and export links, and produced a playable prototype despite some remaining rough edges.

Bug‑Finding by the Evaluator

Rectangle fill tool did not fully fill areas.

Entity spawn points could not be correctly deleted.

Animation‑frame reorder API returned 422 errors.

These issues were critical; without the evaluator the product would have remained a demo.

Ablation Study – Removing the Sprint Layer

In Opus 4.6, Anthropic removed the sprint decomposition, letting the model run a full build before a single QA pass. Planner and evaluator remained because the planner prevents under‑scoping and the evaluator adds value near the model’s capability boundary. They observed that when tasks are comfortably within the model’s stable range, the evaluator adds cost without much benefit.

Complex DAW Experiment

Anthropic built a browser‑based digital audio workstation (DAW) using the Web Audio API. The run lasted ~4 h and cost ~$124. The evaluator identified missing timeline dragging, absent instrument UI, lack of graphical effect editors, stubbed recording, and incomplete clip manipulation. After fixing these, the DAW featured an arrangement view, mixer, and transport, and could generate tempo, melody, drums, and reverb via prompts, though it remains far from a professional tool.

Takeaways

Separating generation from evaluation remains a powerful way to improve agent quality.

Heavy harnesses are only worthwhile near the model’s capability edge; otherwise they add cost and latency.

Effective harnesses require continual load‑bearing analysis—removing or adjusting components as models improve.

In short, long‑duration autonomous coding is not about letting a model run longer; it is about giving it a structured workflow with planning, evaluation, and context resets that together turn code generation into a deliverable product.

Original article: Harness design for long‑running application development – Anthropic Engineering (2026‑03‑24)