LLMs to the Left, Harness Engineering to the Right: Bridging the Gap

After a year of experimenting with Vibe Coding, the author observes a deep gap between raw LLM ability and practical output, attributing it mainly to the surrounding execution environment rather than the model itself. This leads to the core claim: “LLM ← , Harness → .”

Harness Engineering is defined as the control layer that wraps an LLM, handling context organization, tool invocation, permission management, testing, failure recovery, and cross‑session state. An analogy compares the LLM to an engine and the harness to chassis, steering, and brakes—without the latter the vehicle cannot move.

The concept evolves from Prompt Engineering (pre‑2024) focused on crafting effective prompts, to Context Engineering (2025) that builds dynamic information pipelines, and finally to Harness Engineering (early 2024) that formalizes the entire control system.

Empirical evidence shows the impact of a good harness: Nate B Jones reports a jump in programming benchmark success from 42 % to 78 % by swapping only the harness; LangChain’s GPT‑5.2‑Codex runs on Terminal Bench 2.0 improves from 52.8 to 66.5, moving from outside the top 30 to the top 5. A composite‑failure analysis demonstrates that a 95 % per‑step success rate yields only 36 % end‑to‑end success for a 20‑step pipeline, while adding a verification layer raises it to 96 %.

OpenAI’s Codex experiment built a production‑grade app from an empty repo in five months with three engineers, generating ~1 M lines of code, ~1 500 PRs, and a daily average of 3.5 merged PRs per engineer—an estimated ten‑fold speedup over traditional development. The team’s conclusions emphasize a shift from code writing to designing environments, feedback loops, and control systems.

Key architectural practices include keeping the agent’s knowledge explicit in a lightweight AGENTS.md that points to a structured repository (shown in the code block below), using a strict layer hierarchy (Types → Config → Repo → Service → Runtime → UI) enforced by custom linters and CI, and providing agents with Chrome DevTools Protocol access, observable stacks (LogQL, PromQL), and automated garbage‑collection of AI‑generated code drift.

AGENTS.md
ARCHITECTURE.md
docs/
├── design-docs/
│   ├── index.md
│   ├── core-beliefs.md
│   └── ...
├── exec-plans/
│   ├── active/
│   ├── completed/
│   └── tech-debt-tracker.md
├── generated/
│   └── db-schema.md
├── product-specs/
│   ├── index.md
│   ├── new-user-onboarding.md
│   └── ...
├── references/
│   ├── design-system-reference-llms.txt
│   ├── nixpacks-llms.txt
│   ├── uv-llms.txt
│   └── ...
├── DESIGN.md
├── FRONTEND.md
├── PLANS.md
├── PRODUCT_SENSE.md
├── QUALITY_SCORE.md
├── RELIABILITY.md
└── SECURITY.md

Anthropic’s 2025‑11 paper on long‑term agents introduces a two‑stage structure (initializer + coding agents) to maintain continuity across sessions. Cursor’s self‑driving codebases experiments reveal that naïve parallel agents cause state conflicts, leading to a planner‑worker hierarchy where a root planner slices work and workers operate on isolated repo copies.

Limitations are acknowledged: METR’s 2026 study shows that half of AI‑generated PRs passing automatic scoring are rejected by maintainers, and real‑world constraints (security, style, maintainability) still dominate. Noam Brown (OpenAI) warns that harnesses are a temporary crutch, but the underlying architectural constraints will persist, much like steering wheels after the horse‑drawn carriage era.

Final insight: as models become commoditized, the differentiating moat shifts to bespoke harnesses that make AI output repeatable, scalable, and cost‑effective. Therefore, harnesses must remain lightweight, modular, and replaceable.