Why Agents Struggle to Self-Evolve: Three Benchmarks for True Recursive Improvement

Researchers from ByteDance Seed, TokenWave, and affiliated institutions present the Self-Developing Agents project, which argues that current agents can solve well-defined tasks but lack true recursive self-improvement (RSI). They distinguish "half-loop RSI" — where a reliable external verifier guides optimization — from a full closed loop that must also handle goal formation, experience absorption, and persistent system evolution. Three benchmarks target these three gaps.

ASPIRE: Learning from Vague Goals

ASPIRE evaluates whether an agent can turn a natural-language capability direction (e.g., "improve mathematical reasoning") into concrete learning actions without predefined datasets, training recipes, or verifiers. The benchmark provides 6 capability goals, 520 hidden expert-authored test questions (with answers and per-question feedback withheld from the agent), a minimal interactive environment supporting weighted harness updates, and 48 paired "vague goal / explicit task" runs with auditable trajectories.

Experiments show agents spend disproportionate time interpreting goals and selecting proxy metrics, leaving less budget for actual training and evaluation. Across 30 units, 21 produced checkpoints that passed initial screening, but only 1 yield was retained after final validation. LoRA usage rose from 24.1% to 89.8% under vague goals, indicating a shift in search behavior without corresponding goal verification. The core finding: increased decision-making activity does not imply correct goal decomposition, and completing an update does not guarantee target capability improvement.

S³Gym: Self-Testing, Self-Judging, Self-Improving

S³Gym places agents in seven text-based games where they must explore, self-evaluate, and improve without access to ground-truth step rewards or final scores during learning. After exploration, performance is checked under stricter conditions with new random seeds; test trajectories are not fed back as training material.

A key result: self-judgment accuracy does not reliably translate into subsequent performance gains. The correlation between judgment quality and later returns is weak — agents can assess current actions but cannot consistently adjust future actions based on those assessments. Experience storage also lacks a universal solution: raw history and summarized memory each excel in different environments, and the same method can fail when the environment changes. Trajectory analysis suggests some experiences generalize as reusable strategies while others depend on precise state-action details; summarization preserves the former but may lose the latter. Updating model parameters can inject experience but risks catastrophic forgetting — in one run the model improved on a trust game yet regressed continuously on Plants vs. Zombies, with other games unchanged. The cause of regression remains undetermined. S³Gym therefore insists on verifying experience value in downstream actions: review accuracy, storage form, and next-step improvement must be measured separately, not replaced by a plausible-looking summary.

HarnessDev: Persisting System-Level Improvements

HarnessDev lets a developer model create and iteratively modify an execution system (code, ML experiments, writing, research) while a separate, frozen executor model runs tasks. This isolates harness evolution from model-weight changes. The benchmark tracks whether new mechanisms are actually invoked during execution (e.g., whether written memory and state-saving code are called) and evaluates on held-out tasks such as SWE-Pro.

In fixed-executor trajectories with Qwen and DeepSeek as developers, selected versions improved on development-time feedback but both declined on hidden test tasks. Hidden scores were computed only after development ended and never returned to the developer, exposing overfitting to visible feedback. System modifications also broke existing functionality: message cleanup and context compression changes disrupted valid tool-message sequences, forcing rollbacks. HarnessDev emphasizes that persistent changes must be validated for runtime activation, transfer to unseen tasks, and rollback capability. Current evolution observations are primarily from coding tasks, with single-trajectory configurations insufficient to judge long-term stability.

Synthesis: The Bottleneck Is Not "Can It Change" But "Does It Help"

Across ASPIRE, S³Gym, and HarnessDev, agents demonstrate self-training, self-summarization, and self-modification abilities. The real bottleneck is the mismatch between proxy goals and true objectives, and overfitting to local feedback and visible metrics. Future self-developing agents will need more reliable goal-formation, verification, and retention mechanisms to continuously filter effective experience and distill genuinely generalizable improvements into stable capabilities.