Harness Continual Learning: How Agents Evolve Without Model Fine-Tuning

Paradigm Shift: From Model Parameters to Harness Evolution

Traditional continual learning focuses on updating model parameters to acquire new tasks while avoiding catastrophic forgetting. In the Agentic AI era, an agent's behavior is determined not only by the base model but also by its surrounding Harness : system prompts, interaction memory, reusable skills, external tools, workflows, and routing strategies. Even with a frozen base model, the agent can continuously change its behavior by updating these components. Harness Continual Learning (HCL) formalizes this shift, making the Harness the object of continual learning.

HCL Framework: Four Co-Evolving Harness Components

HCL represents the Harness state at stage n as a tuple of four components that jointly determine agent behavior:

Task Interface : Parses and normalizes tasks, defining how the agent understands inputs.

Experience Memory : Stores interaction histories and abstracts transferable patterns.

Capability Map : Manages external tools, perception models, and internal reusable skills.

Adaptive Router : Selects relevant memories and capabilities, and orchestrates execution flows.

This decomposition is a reference design covering common Harness functions; it clarifies what is learned when the model stays frozen.

Harness-Level Forgetting

Harness updates do not always yield positive accumulation. New memories may interfere with retrieval of old knowledge; skill revisions may break valid tool calls; routing or workflow adjustments may cause previously solvable tasks to fail. The paper defines Harness-level Forgetting as the degradation of previously reliable behaviors caused by Harness updates, manifesting as correct answers turning wrong, tool calls failing, or action trajectories unable to achieve original goals.

Guarded Harness Evolution: Proposal–Evaluation–Commitment Loop

To decide which updates deserve long-term deployment, HCL separates generating modifications from deploying modifications via a three-stage loop:

Proposal : A Continual Optimizer analyzes the current Harness, raw interactions, execution results, and feedback to diagnose whether issues stem from Interface, Memory, Capability, or Router. It generates candidate modifications for the relevant component(s). Candidate states are isolated from the deployed state until evaluation finishes.

Evaluation : A Continual Evaluator checks candidates on three criteria: (1) improvement on the current task; (2) preservation of reliable behavior on historical anchor points (anchors are used only for evaluation, not exposed to the Optimizer to prevent overfitting); (3) validity of output format, tool calls, and environment actions. A historical loss tolerance b controls the maximum allowed regression on anchors.

Commitment : Only candidates satisfying all three criteria become the next deployed Harness version; otherwise the system retains the previous version.

Relation to Traditional Continual Learning

HCL retains the core continual learning objective—acquiring new capabilities while retaining existing ones—but extends it to the Harness layer. It unifies multiple traditional ideas within one system: Experience Memory preserves historical experience; abstracted memory and Capability Map support knowledge transfer and capability reuse; the Continual Optimizer provides plasticity for new tasks; the Continual Evaluator enforces stability via historical constraints. Rather than porting a single algorithm, HCL extends the fundamental continual learning mindset to the agent system level.

Experimental Validation: Frozen Models, Evolving Harness

All experiments keep model parameters frozen; performance changes come solely from Harness evolution.

ALFWorld (Embodied Tasks)

Static Harness final average success rate: 47.12%. Stability-HCL: 61.74% (average forgetting 2.64 percentage points). Plasticity-HCL: 62.98% (average forgetting 10.94 pp). Similar final performance but vastly different forgetting, showing update constraints critically affect historical loss.

Minecraft (Long-Horizon Open World)

Static Harness stops progressing after task 15. HCL completes all 50 tasks. HCL uses 83 environment actions, fewer than MemRL (88) and MemP (91), indicating Harness evolution enables both longer task progression and higher execution efficiency.

Text Reasoning Task Streams

Frozen DeepSeek-V4-Flash zero-shot final average: 45.50%. Plasticity-HCL raises it to 64.70% with average forgetting of only 0.07. GSM8K jumps from 49.40% to 92.00%.

Multimodal Perception Task Streams

Frozen Qwen3.6-27B zero-shot final average: 39.40%. Baseline DGG: 42.73%. Stability-HCL reaches 68.92% with average forgetting of 0.22 pp.

These results confirm that agents can continuously accumulate capabilities via Harness evolution alone, but different update strategies lead to different forgetting levels.

Stability–Plasticity Trade-off in Harness Evolution

Varying only the historical loss tolerance b reveals a non-monotonic relationship: as constraints relax, forgetting increases steadily, but final performance peaks at a medium tolerance and then declines. Stricter historical constraints consistently suppress forgetting across the task sequence (Figure 4). This shows that higher update freedom does not guarantee better long-term outcomes; the key is selecting modifications that serve the long-term evolutionary trajectory.

Component ablation shows the same trade-off: full HCL achieves highest final average performance; freezing individual Harness components sometimes reduces forgetting but also limits new capability acquisition.

Conclusion: Continual Learning at the Agent System Level

HCL reframes many agent system design questions as continual learning problems: how to retain expanding historical behaviors, how to attribute interference across Harness components, and how to control evaluation and update costs over long horizons. It challenges the community to re-answer three fundamental questions: What is learning? What must be retained? How can current updates serve long-term learning? HCL provides a unified framework to formally define, measure, and study capability acquisition, behavior retention, and forgetting in Harness evolution, offering a new research perspective for moving continual learning from model-level to agent-system-level.

Paper: Harness Continual Learning: Continual Adaptation Beyond Model Parameters (https://arxiv.org/pdf/2608.19013) (Nanjing University Reasoning & Learning Group, University of Wollongong).