Harness Engineering: The Deterministic Layer That Makes AI Agents Stable and Efficient

OpenAI introduced the concept of Harness Engineering in February 2024. Harness, literally "horse tack" (reins, saddle, protective gear), is a tool or mechanism that lets the "horse" (the model) run better and more stably. Its core demand is to make the model "hit where you aim."

When You Need Harness — And When You Don't

The author outlines three scenarios where Harness is not necessary:

Scenario 1: Short-term, ad-hoc, low-volume single-turn Q&A or retrieval — model + system prompts + RAG suffices; Harness only adds complexity. Input → Process → Output Harness❌ Scenario 2: Unlimited budget — just use the strongest model; Harness may lower your ceiling. Ample Budget Harness❌ Scenario 3: Fully AI-native architecture where core value sits directly on model reasoning/generation/decision — a small team can match big-company output. Full AI Native Harness❌ Outside these, Harness is necessary. The key factors: task complexity and reliability/consistency requirements; cost-effectiveness (many ways to complete a task, but only cost-appropriate ones scale); and whether the agent is an autopilot or a copilot .

Why Harness Engineering?

The author's team wanted the model to detect malicious attacks, assess risky requests, and manage team memory. Concrete pain points:

Detect malicious attacks: Gateway logs reach terabytes per day; even after cleaning, still near TB scale. Feeding this directly to the model explodes context, cost, and latency.

Assess risky requests: Without MCP tools the model has no "eyes"; with tools, where should its attention focus? Must we remind it every step?

Manage team memory: What deserves long-term memory? How to migrate memory when extending the agent platform?

As team agent usage grew, problems emerged: context explosion, task timeouts, hallucinations, permission management. Prompt optimization alone couldn't fix them. The author needed to constrain agent boundaries, redesign task scheduling, and manage context — this deterministic layer outside the model is Harness Engineering.

Agent = Model + Harness. Model handles cleverness; Harness handles stability.

How to Do Harness Engineering: Four Pillars

01 Constrain Action Space (Tools / Skills / Permissions / Retry & Circuit Breaker)

External Tools: skill & MCP

Agents run in sandboxes for data security. First step: give the agent "sight" and "reach" to real data. For small data, manual copy-paste works; for recurring tasks, build stable, reusable skills and MCP tools. Solidified tools improve model cache hit rates and lower cost.

Skills are not "the more the better." Community reports show 10,000+ skills degrade agent performance. Add skills iteratively, problem-driven. Examples: idp-mcp for Hive queries, ES query/analysis skills for real-time traffic, threat-intel skills for IP reputation. These atomic capabilities give the model ready-made tools, avoiding ad-hoc reasoning each time, and are reusable across agents.

02 Manage I/O & Context (Layered Dimensionality Reduction, Cost)

1) Data Precomputation

Don't feed low-information-density data to the model; let the model do high-value reasoning and attribution.

The team's data warehouse has three layers: ods (raw logs) — TB scale dwd (cleaning, feature extraction) — TB scale dws (statistical dimensions) — ~100 MB/day

ODS and DWD are unsuitable for direct model analysis. Outside the agent, cleaning and dimensionality reduction are applied. At DWS, daily data is ~100 MB; partitioned by hour it becomes a few MB — trivial for the model. DWS already contains processed dimension/fact metrics. DWD detail data still serves for fine-grained drill-down when DWS flags anomalies.

2) Cost Control

Reduce context pressure.

After giving the agent atomic tools, we direct it to complex tasks. We want more data and longer runs, but attention drift lowers efficiency and tokens aren't infinite. The model must make attention trade-offs.

Passive limits (external):

Tool layer: MCP tools return limited rows, session connection timeouts

Model layer: thinking rounds, thinking duration

Cost layer: token plan

Active intervention (internal):

Control queries: skill docs specify time range, query scope, depth; prompts enforce them

Hierarchical analysis: high priority → full analysis; medium → Top-N; low → sampling

Sampling analysis: low priority sampled

Covering 1,000+ APIs, the model can't scan all traffic. Methodology: pre-compute risk levels on DWD using cheap resources, then hierarchical analysis — high risk + ES full query, medium/low risk → hierarchical analysis.

03 Scheduling & Triggering ("When to Work" Handed to Deterministic Systems)

1) Instruction System

Simplest approach: inject prompts at system level (e.g., CLAUDE.md, ./prompts) with clear definitions. Once defined well, avoid frequent changes — changes destabilize output; one-time definition yields highest ROI.

duties.md example:

Work mode: receive question → think if tool needed → make analysis plan → execute tools → structured answer

Presentation: structured, agreed format

Principles: which tools, solution comparison, info sources

Prohibitions

rules.md: Prevent intermediate files cluttering workspace; keep workspace clean.

Boundary constraints: "Stay silent unless @-mentioned." Hallucinations are hard to eradicate even via memory; edge-case constraints declared explicitly in rules.md.

2) Task Scheduling

Agent should not decide "when to work"; scheduling belongs to external deterministic systems. Analysis, reasoning, attribution go to the model.

Cron jobs cause duplicate runs, missed runs, timeouts — low success rates. Tried A2A calls (early unsupported) and scheduled data probes — partial fix.

Switched to event-driven triggering: upstream task completion fires an event that invokes the model. Model no longer guesses upstream readiness.

04 State & Memory Persistence (Memory, Git Versioning)

1) Memory Management

Humans act as reviewers during interaction, contributing business judgments, preferences, workflows — private-domain info. Without this pre-accumulated memory, the human-model "break-in period" is long. For stable performance, start memory management from day one as a long-term asset.

2) Version Control

Use git to manage agent base files: system prompts, memory files, startup scripts. Ensures traceability, team sharing, sandbox rebuild without loss.

Operational Metrics Comparison

Same model ( qwen3.7-plus), same time window, before/after Harness adjustments:

Scheduling: Event-pipeline driven, no cron. Agent no longer manages its own schedule state → fewer tool calls.

Retry & circuit breaker: New rule: "max 2 retries per operation, don't keep trying different param formats." Previously, token explosion and long runs came from model obsessing over tool-call failures.

Query pattern: Changed repeated queries to one data snapshot export to workspace, then local aggregation — eliminated repeated Hive reads/writes and error retries.

Logic solidified into scripts: Script-first; high-consistency tasks become stable scripts/commands, preventing inconsistent calibrations and saving model inference time.

Result: Agent Loop significantly shortened, stability improved, execution efficiency up. Overall token consumption rose, but mostly from cache reads/writes. Cache hit price is 1/10 of regular input price — small cost for large efficiency gain.

Fundamentally, back to the opening principle: Let the model focus on solving the problem, not figuring out the problem.

Open Problems & Outlook

Beyond keeping the model running stably, an agent that only outputs without human feedback may self-reinforce into a "dead end." Next focus: collect real human feedback/corrections, let the agent summarize lessons to avoid repeating mistakes.

References: 1. Harness Engineering for Self-Improvement 2. https://openai.com/index/harness-engineering/