What Is Harness Engineering? The Missing Piece Behind Stable AI Agents

What Is Harness Engineering?

Harness Engineering is the discipline that makes an AI Agent behave reliably in real‑world, long‑running tasks. It is captured by the equation

Agent = Model + Harness

where the model provides raw intelligence and the harness supplies the surrounding infrastructure.

From Prompt to Context to Harness

AI development has progressed through three stages. Prompt Engineering teaches the model to understand a task (the "what"). Context Engineering ensures the model receives the right information (the "where"). Harness Engineering adds the missing "how"—the orchestration, state management, evaluation, and recovery that keep the agent on track over many steps.

Six Layers of a Mature Harness

Context Refinement : decide exactly what the model should see in each call.

Tool System : expose safe, well‑defined tools (e.g., Git CLI, Slack API) and decide when to invoke them.

Task Orchestration : define a clear execution flow (ReAct, Plan‑and‑Execute, etc.).

Memory & State : externalise long‑term state to files so each turn starts with a clean context.

Evaluation & Observation : maintain an Eval set and trace logs to measure success.

Constraints & Recovery : hard‑code safety rules, validate outputs, and define automatic retry or fallback paths.

Concrete Example: PR Review Agent

The article walks through a practical agent that each day scans GitHub pull requests, scores their importance, generates a summary, and posts the result to Slack.

1. 拉取仓库当前所有开放的 PR 列表
2. 对每一个 PR:
   a. 读取 diff 与描述
   b. 判断是否涉及核心模块
   c. 对核心 PR 做深度分析并打分 (1‑5)
3. 按分数排序，选前 3 条
4. 为每条生成摘要和点评
5. 汇总并发送到 Slack
6. 检查发送成功，失败则重试

Key decisions include:

Only the minimal, task‑relevant information (title, changed files, module docs, author style) is fed to the model each turn.

Four tools are provided: gh CLI, file reader, code search, and Slack sender.

State such as processed PR IDs and daily progress is stored in today‑progress.json and re‑loaded on each iteration (Context Reset).

After each step the output is validated (e.g., Markdown format) before proceeding.

Principles Extracted from Industry Practice

Restart > patch: when the model shows "context anxiety" (its own sense of running out of window), discard the old context and start a fresh turn, externalising state to files.

Separate production and evaluation: the Planner (produces code) and the Evaluator (runs real tests) must be distinct agents.

Improve the environment, not the model: add linters, unit tests, and a sandbox so the agent can self‑correct rather than relying on stronger prompts.

Progressive disclosure for rules: keep a short index file (≈100 lines) and load detailed policy documents only when needed.

Continuous automated debt repayment: encode "Golden Principles" as code‑style rules and run background agents that automatically open fix‑PRs for any violation.

Common Pitfalls and How Top Companies Solve Them

Context anxiety: agents become overly cautious as the window fills. Solution – Context Reset with external state files (Anthropic).

Self‑evaluation bias: letting the same agent grade its work leads to over‑optimism. Solution – separate Planner, Generator, and Evaluator roles (Anthropic).

State loss across turns: without externalised memory the agent repeats work. Solution – persist progress in JSON/YAML files and reload each turn.

Rule‑file bloat: long monolithic AGENTS.md causes context corruption. Solution – split into a concise index and modular sub‑documents (progressive disclosure).

AI slop (code quality decay): agents copy bad patterns. Solution – codify best‑practice rules, run automated lint/CI agents, and let them open corrective PRs daily.

Why Harness Matters Now

As models plateau, the biggest gains come from engineering the surrounding system. A well‑designed harness turns a powerful language model into a production‑grade AI assistant that can reliably execute multi‑step workflows, self‑heal, and continuously improve.

Agent = Model + Harness

Investing in Harness Engineering—building the six layers, adopting the five principles, and avoiding the five common pitfalls—offers the most practical path to stable, scalable AI agents today.