Harness-S: Internalizing Complementary Harness Experience for Generalizable Agents

Research Motivation: Why Harness Diversity Matters

An agent's success depends on both model capability and its execution framework (Harness) — the prompts, tool interfaces, context organization, and execution rules surrounding the model. Improving the harness can turn previously failing tasks into successes, raising the question: can the abilities elicited by these framework improvements be retained in the model itself through training, so they persist when the model returns to the original harness?

The paper identifies three key properties of harness variants evolved from a common base (DeepSeek Harness):

Locality : A fix helps on some tasks but may degrade others; stacking fixes into one harness creates interference.

Complementarity : Different variants solve different tasks; jointly they cover more successes than any single variant.

Nonstationarity : A variant's usefulness changes as the model learns; what helps early may become unnecessary later.

These properties motivate preserving a pool of complementary sibling harnesses and adapting their use as the model evolves.

Method Framework: Harness-S

Harness-S consists of three modules that convert complementary harness expertise into shared policy learning:

Directional Harness Evolution

Starting from the base harness H₀, the method analyzes failure trajectories, groups failure causes into distinct repair directions, and generates child variants for each direction. Parent selection for the next evolution round maximizes joint validation coverage — the union of tasks solved by a set of candidates — ensuring diverse branches survive. All executable, protocol-compliant variants are retained in a Sibling Harness Pool , which remains fixed during policy training.

Complementary Harness Rescue Training

For each task, the current policy first generates K trajectories under H₀. If any succeed, that group is kept. If all fail, the Adaptive Harness Router selects K distinct auxiliary variants from the pool; the same policy re-executes the task from the initial state under each chosen variant, producing a rescue trajectory group. This rescue group replaces the all-failed group for policy updates.

Because all variants share the same environment interface, tool calls and task operations valid under an auxiliary harness are also executable under H₀. The auxiliary context merely guides the policy to explore effective behaviors that are hard to sample under H₀ alone. Reinforcement learning (GRPO) then updates the shared policy using the task rewards, turning context-supported successes into stable behavioral tendencies.

Adaptive Harness Router

The router maintains Harness Behavior Profiles describing when a variant helps or hurts. For a given task, multiple LLMs independently rank candidate variants based on profiles, validation performance, and similar-task outcomes; rankings are aggregated via weighted voting (weights decay with rank). The top-K distinct variants (excluding H₀) form the rescue set.

Profiles are periodically re-anchored: at checkpoints, the current policy is frozen, run on all variants and H₀ over the validation set, and new profiles are generated from the paired execution results. This adapts routing to the changing utility of each variant as the policy improves.

Main Results: Gains Retained in the Base Harness

Experiments use DeepSeek Harness as H₀, Qwen3.5-4B and Qwen3.5-9B, across AppWorld (API operation), SWE-Gym (software engineering), and DBBench (database) benchmarks. Metrics: Pass@3 (fraction of tasks solved in ≥1 of 3 attempts) and Avg@3 (mean success rate over 3 attempts).

On both model sizes and all three benchmarks, Harness-S trained models evaluated back on H₀ outperform standard GRPO and an H₀-resampling rescue baseline. For 4B on AppWorld, Pass@3 rises from 60.31% (GRPO) to 67.56%; on SWE-Gym from 24.38% to 38.75%.

Against the H₀-resampling baseline (same failure-triggered extra attempts but staying in H₀), Harness-S achieves higher Pass@3 and Avg@3 across the board, confirming that complementary execution paths provide extra learning signal.

After training, task-level Rank-1 routing (choosing one harness per task) further improves all metrics; a routed 4B model even surpasses a routed 9B model trained with H₀-resampling rescue.

Learning Mechanism: How Complementarity Becomes Training Signal

The paper measures Effective Groups — trajectory groups containing both success and failure — because only they yield non-zero relative advantages for GRPO. On AppWorld with 4B, Harness-S maintains a higher effective-group proportion throughout training (79.38% vs. 70.94% for H₀-resampling) and a higher fraction of policy tokens participating in effective groups. This shows the gain comes from richer reward contrasts, not just more trajectories.

Fixed mixtures of multiple harnesses (e.g., H₀ + Claude Code, or H₀ + several sibling harnesses) achieve effective-group rates similar to plain GRPO, indicating that timing matters : concentrating complementary support on tasks where H₀ already fails creates the needed success/failure contrast.

Routing Ablation: Selection and Adaptation Both Matter

On AppWorld 4B, with equal rescue attempt budgets:

Adaptive routing (updated profiles) outperforms fixed profiles: rescue success rate higher at every training stage, final Avg@3 +4.71 percentage points.

Broader exploration helps: assigning one attempt each to 8 distinct variants beats two attempts each on 4 variants, yielding better rescue rates and final test performance.

These results confirm that re-evaluating variant usefulness as the policy learns, and preserving diverse repair specializations, both contribute materially.

Behavior Transfer: Concrete Skills Retained in H₀

Qualitative analysis on H₀ after Harness-S training shows the model adopts reusable practices:

More frequent API schema lookup before first call.

Fewer repeated code executions.

More pagination queries and post-write state checks.

In SWE-Gym: more assertions, re-verification of fixed targets, and pre-submission diff checks.

Two case studies illustrate generalization to unseen test instances:

Collection filtering : Training rescue used a variant that correctly enforced intersection constraint ("liked AND in library/playlist"). On a held-out test task, GRPO took the union (over-fetching), while Harness-S preserved the intersection and sent the correct list.

Account recovery : Training rescue completed a multi-step Venmo flow (read code, reset password, verify login). On an unseen Amazon verification task, GRPO stalled after reading email; Harness-S continued to reset password and verify new credentials.

Routing behavior also shifts: H₀ becomes the router's top choice more often, and the Rank-1 vs. H₀ performance gap narrows, consistent with the model internalizing auxiliary-harness-guided behaviors.

Cross-Framework Generalization

Without further parameter updates, the Harness-S trained 4B model is evaluated directly under Claude Code and OpenClaw on AppWorld and SWE-Gym. It achieves the highest Pass@3 in every framework-task combination, including beating GRPO trained with fixed harness mixtures. Absolute performance remains lower than under H₀, showing that while learned behaviors transfer across execution interfaces, the specific context and support still affect reliability.

Research Implications

Harness-S demonstrates a path to turn engineering effort on execution frameworks into reusable model capabilities. By evolving complementary harness variants, directing their use to where the base harness fails, and continuously adapting the selection to the current policy, diverse framework improvements become training resources for a single model. Future work includes broader framework reuse, long-term transfer across policies, and analyzing the trade-off between diversity gains and profiling costs.