How Harness Puts a ‘Bridle’ on AI: Keeping Programming Agents Under Control

Overview

The previous piece introduced the Agent Loop, where an AI follows a plan‑act‑observe‑reflect cycle to complete coding tasks. The Loop defines the behavior pattern, but running it reliably requires Harness, an execution engine that schedules the loop, manages tools, enforces policies, and maintains state.

Problems Harness Solves

Without Harness, the LLM outputs code directly with no verification; with Harness, files are read first, then modified, then tested.

Unrestricted tool calls can execute dangerous commands such as rm -rf; Harness uses a whitelist and requires approval for risky actions.

Loops can run indefinitely, consuming tokens; Harness imposes step limits and automatic time‑out termination.

Repeated failures repeat the same operation; Harness detects duplicate failures and switches strategy automatically.

The process is untraceable; Harness records a complete Plan/Act/Observe trace for every round.

Context can grow without bound; Harness automatically compresses history while preserving key information.

Relation Between Harness and Agent Loop

Agent Loop is the behavioral model; Harness is the runtime environment. The Loop tells the agent what to do each step, while Harness decides whether the step is allowed, how to execute it, and what to do afterward.

Full Path of One Loop Iteration in Harness

Policy Layer : Check whether the current step exceeds limits and whether the requested tool is permitted.

Loop Scheduler : Feed context to the LLM and obtain Plan and Act decisions.

Tool Layer : Execute file reads, command runs, etc., and collect Observe results.

State Management : Record the trace, compress context, and decide whether to terminate.

If not terminated, return to step 1; if human intervention is required, pause and wait.

Harness intervenes on every Loop round rather than only after the Loop finishes.

Core Components of Harness

1. Model Layer

Handles interaction with the LLM: sending prompts, receiving responses, and parsing structured output.

Select appropriate model (reasoning vs. coding).

Control temperature, max tokens, etc.

Parse function‑calling / tool‑use request formats.

2. Tool Layer

Provides the Act‑stage capabilities of the Agent Loop.

Register available tools and their schema.

Execute tool calls and return structured results.

Handle tool failures, timeouts, and retries.

3. Policy Layer

The "bridle" that defines what the agent may or may not do.

Permission control : which files can be read, which commands can be executed.

Step and timeout limits : maximum Loop iterations, per‑tool timeout, overall task timeout.

Approval rules : actions such as file deletion, code push, or production access require human confirmation.

Duplicate detection : after N consecutive identical failures, force a strategy change or abort.

The design of the Policy Layer directly determines the safety boundary of the agent.

4. State Management

Maintains the complete runtime state of the agent.

Conversation history : full record of each Plan/Act/Observe round.

Task progress : current goal, completed steps, pending items.

Context window : dynamic trimming and compression to avoid token overflow.

Checkpoints : support pause, resume, and rollback to a specific Loop node.

State management gives the agent memory and traceability, enabling post‑mortem analysis and task continuation.

5. Human Interface

Defines how humans intervene in the Loop.

High‑risk operations trigger confirmation dialogs.

The agent can request additional information.

Users may pause, correct direction, or cancel the task at any time.

After completion, a change summary is presented for review.

Harness does not exclude humans; it inserts them at the right checkpoints.

Running Example

Task: "Add email‑format validation to UserService".

Harness 启动
  ├─ 策略层：加载项目权限（可读写 src/，不可执行 deploy）
  ├─ 状态管理：初始化任务目标、步数计数 = 0

Loop 1
  ├─ 策略层：步数 1/20，通过
  ├─ LLM 决策：Plan → 先读取 UserService 源码
  ├─ 工具层：read_file("src/services/UserService.ts")
  ├─ 状态管理：记录轨迹，步数 = 1

Loop 2
  ├─ LLM 决策：Plan → 编写校验逻辑 + 单元测试
  ├─ 工具层：edit_file(...) + write_test(...)
  ├─ 状态管理：记录变更

Loop 3
  ├─ LLM 决策：Plan → 运行测试验证
  ├─ 工具层：run_command("npm test -- UserService")
  ├─ Observe：3 个测试全部通过
  ├─ 状态管理：标记任务完成

Harness 终止
  └─ 输出：变更摘要（修改 1 个文件，新增 1 个测试文件，测试通过）

If Loop 3’s tests fail, Harness does not finish; it feeds the failure back to the LLM, enters Loop 4 for repair, and repeats until success or the step limit is reached.

Design Principles

1. Controllable

Every agent action is bounded by pre‑set policies; disallowed behavior is blocked at the policy layer rather than relying on the LLM’s self‑restraint.

2. Auditable

The full decision chain and execution trace are recorded, allowing any code change to be traced back to the specific Loop round, observation, and reasoning.

3. Interruptible

Users or systems can pause, cancel, or redirect the agent at any Loop node, preventing the agent from becoming an opaque black box.

4. Fail‑Closed

When the policy layer encounters uncertainty, it defaults to denial and may request human approval instead of assuming safety.

5. Incremental Verification

Harness encourages small, frequent actions and observations rather than large, monolithic changes; policies can limit the scope of each modification to enforce this behavior.

Common Anti‑Patterns

1. Bare‑run Calls

Sending raw user requests directly to the LLM without Harness leads to unverifiable output, loss of control, and unrecoverable errors. Mitigation: at minimum enforce step limits, tool permissions, and basic trace logging.

2. Over‑Permissive Policies

Having Harness but leaving permissions essentially open defeats its purpose; the agent can execute any command, risking irreversible damage. Mitigation: apply the principle of least privilege, deny by default, and grant permissions only as needed.

3. Over‑Restrictive Policies

Setting step limits too low or requiring approval for every action causes frequent interruptions, low efficiency, and poor user experience. Mitigation: tier risk—allow low‑risk actions automatically, reserve approvals for high‑risk operations.

4. Stateless Execution

Discarding trace information each round prevents debugging, replay, and causes context overflow on long tasks. Mitigation: establish state management and trace recording from day one.

5. Mixing Harness and Loop Responsibilities

Embedding constraints like "do not loop more than 10 times" in the prompt rather than in the policy layer makes them unenforced. Mitigation: place hard constraints in Harness’s policy layer and use prompts only for soft guidance.

Practical Takeaways

Build Harness first, then tune prompts; reliability comes from framework constraints, not from the model’s self‑discipline.

Minimize permissions so the agent only receives the tools required for the current task.

Implement tiered approvals: auto‑allow reads, log writes, and require confirmation for deletions or deployments.

Log everything; trace logs are the only reliable way to debug agent issues.

Set reasonable boundaries: step limits must balance safety and efficiency.

Support interruption and resume; long‑running tasks must survive network hiccups without losing progress.