How SEAGym Tackles Evaluation Challenges of Self‑Evolving LLM Agents

LLM agents increasingly improve by evolving external components—prompts, memory, tools, workflow, middleware, permission control, and runtime state—collectively called the agent harness. Static benchmarks that fix an agent and report a final success rate cannot reveal which harness parts change, whether improvements transfer to unseen tasks, whether they over‑fit recent feedback, cause forgetting, or increase cost or instability.

SEAGym: An Evaluation Environment for Self‑Evolving LLM Agents

SEAGym formalises a self‑evolving agent as a sequence of snapshots. Each snapshot consists of a fixed base model M and an updatable harness state H, where H includes prompts, memories, skills, tools, middleware, and runtime configuration. At each step the environment samples a batch of training tasks, the agent executes them, and the system records trajectories and feedback. The agent then modifies its harness via a unified rollout/update interface. No specific update algorithm is imposed, allowing any self‑evolution method to be plugged in under the same protocol.

Multi‑View Evaluation Design

Train batch : provides trajectories and feedback for updates.

Update‑validation : freezes intermediate snapshots to measure stage‑wise improvement.

ID transfer : tests generalisation to unseen in‑distribution tasks.

OOD transfer : tests generalisation to out‑of‑distribution tasks.

Replay : re‑executes old tasks to detect forgetting or regression.

Cost records : logs token usage, tool calls, runtime time, and update cost.

This fine‑grained reporting exposes phenomena hidden by a single leaderboard score, such as temporary performance drops, middleware failures, or runtime contract violations.

Experimental Setup

Terminal‑Bench 2.0 : execution‑heavy tasks involving command‑line operations, software‑engineering, and environment interaction.

HLE : reasoning‑heavy, text‑only Math/Physics tasks; CS/AI and Engineering tasks are used as OOD transfers.

Three self‑evolution methods are evaluated:

ACE : accumulates prompt‑visible skillbooks and procedural experience.

TF‑GRPO : updates an experience/context store using grouped rollout evidence.

AHE : directly edits a broader harness (prompts, tools, middleware, runtime behaviour).

Key Findings

Result 1 – Validation gains do not guarantee stable generalisation :

AHE improves validation from 40.0 → 57.1 (+17.1 pp), ID from 40.0 → 49.1 (+9.1 pp), OOD from 22.5 → 28.8 (+6.3 pp).

ACE yields modest gains: validation +2.9 pp, ID +3.6 pp, OOD +2.5 pp.

TF‑GRPO shows a large validation boost (+17.1 pp) but a drop on OOD (‑2.5 pp), indicating over‑fitting to the source distribution.

Result 2 – Self‑evolution can cause intermediate crashes :

In AHE’s train‑replay experiment the agent solves 34/80 training tasks initially and 43/80 finally, but snapshot 4 suffers a severe drop to 6/80 due to middleware contract violations; later updates recover performance.

The failure manifests as broken tool‑path, middleware contract, or runtime protocol rather than pure knowledge loss.

Result 3 – Batch size affects harness stability (batch sizes 10, 20, 40, 80):

Batch 10: validation 37.1 → 22.9, ID 38.2 → 23.6.

Batch 20: validation 40.0 → 57.1, ID 40.0 → 49.1 (best balance).

Batch 40: validation 37.1 → 40.0, ID 41.8 → 43.6.

Batch 80: validation 42.9 → 25.7, ID 41.8 → 25.5.

The non‑monotonic pattern shows that too small a batch provides insufficient evidence, while too large a batch dilutes per‑task focus, both leading to instability.

Result 4 – Training source diversity improves recovery :

Mixed‑source training (Terminal‑Bench + HLE) prevents final collapse.

HLE‑only training achieves temporary gains but ends with validation, ID, and OOD all dropping to 0, despite an intermediate snapshot that still gains ID +7.3 pp and OOD +3.8 pp.

These findings suggest that a single benchmark can push the harness toward a benchmark‑specific local optimum, whereas diverse sources provide complementary signals (tool/runtime errors vs reasoning errors) that aid recovery.

Result 5 – Harness updates depend on the backend model (cross‑model transfer with DeepSeek, GLM, GPT‑5.4):

DeepSeek‑evolved harness on DeepSeek: ID +9.1 pp.

GLM‑evolved harness on GLM: ID +3.6 pp.

GPT‑5.4‑evolved harness on GPT‑5.4: ID +5.5 pp.

Cross‑backend asymmetries: DeepSeek‑evolved harness gives GLM +7.3 pp but harms GPT‑5.4 (‑3.6 pp); GPT‑5.4‑evolved harness helps GPT‑5.4 (+5.5 pp) but harms GLM (‑7.3 pp). OOD gains are often zero or negative.

Thus harness modifications are not universally transferable; different roll‑out models expose distinct failure surfaces (tool recovery, reasoning, artifact constraints), leading to backend‑specific benefits.

Significance and Future Directions

SEAGym shifts evaluation from “final success rate” to “which harness components change, how they generalise, whether they cause forgetting, and at what cost”. This process‑level insight is crucial for agents deployed in software engineering, data analysis, and long‑horizon tasks where hidden regressions can jeopardise reliability and safety.

Future work includes extending SEAGym to web/desktop interaction, multi‑agent collaboration, continuous online streams, and improving evaluation efficiency through adaptive snapshot selection and budget‑aware replay strategies.

Paper: https://arxiv.org/abs/2606.17546

Code: https://github.com/antropy-research/SEAGym