MemoHarness: The Next Evolution of Agents Happens Outside the Model

Agent Harness Concept

MemoHarness introduces the notion of an Agent Harness , a control layer that determines what context the model sees, which tools it may call, how many reasoning steps are performed, what memory is retained, and how the final answer is extracted and verified. This moves the focus of agent improvement from internal model weights to the surrounding system.

Six Editable Dimensions

The system is decomposed into six controllable dimensions:

D1 Context Assembly : construct input with instructions, constraints, retrieved materials, and examples.

D2 Tool Interaction : decide when and how to invoke external tools or retrieval modules.

D3 Generation Control : adjust decoding parameters such as max tokens, temperature, and sampling.

D4 Task Orchestration : choose between single‑call or multi‑stage “plan‑execute‑refine” workflows.

D5 Memory Management : preserve key state across calls, summarize trajectories, and discard stale context.

D6 Output Processing : extract answers, validate format, and apply fallback strategies on failure.

Turning Traces into Experience

During a development phase, MemoHarness runs many candidate Harness configurations on tasks with reference answers. Each run records task features, the Harness version, the delta from the previous configuration, the full model‑tool trace, the result score, token consumption, and diagnosed failure modes. These per‑case records form an experience database. In a separate testing phase, the frozen experience base is queried to adapt a global Harness to a new, unlabeled task in a single pass.

Experimental Evaluation

The authors evaluate three task families:

Terminal‑Bench (long‑horizon terminal operations): score improves from 0.722 to 0.806.

LiveCodeBench (single‑shot code generation): score improves from 0.900 to 0.967.

FinanceAgent (multi‑step financial analysis): score improves from 0.600 to 0.767.

Cross‑task transfer is selective: the Terminal‑Bench‑derived Harness helps on MMMLU, StrongReject, and SWE‑Bench Pro but not on HumanEvalFix or Reasoning‑Gym‑Easy. Across six additional models, applying the searched Harness yields an average success‑rate gain of +0.098 (range +0.038 to +0.233), suggesting some strategies are model‑agnostic.

Cost analysis on Terminal‑Bench shows 14.18 M input tokens (13.32 M cached) costing $6.89 under public pricing, cheaper than Codex/Claude Code but higher than OpenCode, with the advantage hinging on high cache reuse.

Limitations and Future Work

The paper acknowledges several constraints: limited experimental scale, absence of confidence intervals, non‑uniform baselines, and incomplete ablations of the experience components. The current system does not continuously ingest new experience after deployment; test‑time adaptation is a one‑shot process, and online lifelong evolution is left as future work. A companion study (Wang et al., 2026) stresses the need for rigorous baselines that separate search effort from genuine improvement.

Implications for Engineering

Because the underlying model remains frozen, every Harness modification can be versioned, benchmarked, and rolled back, offering enterprises a testable, controllable, and cost‑aware “operating system” for agents. The accumulated experience becomes a reusable asset that can differentiate organizations even when they share the same base model.

Agent evolution may no longer be limited to the model; the surrounding system can evolve instead.

Reference: Huang et al., “MemoHarness: Agent Harnesses That Learn from Experience,” arXiv:2607.14159 (2026); Wang et al., “Rethinking the Evaluation of Harness Evolution for Agents,” arXiv:2607.12227 (2026).