How SEAGym Enables Self‑Evolving LLM Agents and Solves Evaluation Challenges

Wen Li’s recent article reviews harness engineering for AI self‑improvement, arguing that the harness layer—comprising prompts, memory, tools, workflow, middleware, permission control, and runtime state—has become as crucial as the base model for planning, tool invocation, context management, state saving, and result evaluation.

The shift in LLM agent research is that capability gains now stem not only from model parameter updates but also from the evolution of external harness components. This raises a key evaluation problem: how to benchmark agents that continuously modify their own harness.

Existing static benchmarks fix an agent and report final success rates, which cannot answer whether a harness update improves specific abilities, generalizes to unseen tasks, over‑fits recent feedback, forgets old skills, or incurs higher runtime cost.

To fill this gap, a team from Tsinghua University proposes SEAGym: An Evaluation Environment for Self‑Evolving LLM Agents . SEAGym does not prescribe a particular update algorithm; instead, it defines a unified rollout/update interface that any self‑evolution method can plug into.

Each agent snapshot is represented as M (the immutable base model and runtime components) plus H (the mutable harness state, including prompts, memories, skills, tools, middleware, and runtime configuration). In every step, the environment samples a batch of training tasks, the agent executes them, generates trajectories and feedback, and then updates its harness according to its own rule.

SEAGym separates the training data split from the evaluation view, offering multiple evaluation perspectives:

Train batch : provides trajectories and feedback for harness updates.

Update‑validation : freezes intermediate snapshots to check whether updates yield stage‑wise improvements.

ID transfer : tests whether updates transfer to in‑distribution but unseen tasks.

OOD transfer : tests transfer to out‑of‑distribution tasks.

Replay : re‑runs old tasks to detect forgetting or regression.

Cost records : logs token usage, tool calls, runtime, and update cost.

These views expose fine‑grained dynamics such as a snapshot that improves validation but degrades OOD performance, or a temporary strength spike that later crashes due to a faulty middleware change.

Experiments instantiate SEAGym on two complementary task sources: Terminal‑Bench 2.0 (execution‑heavy, command‑line and software‑engineering tasks) and HLE (reasoning‑heavy, text‑only Math/Physics tasks, with CS/AI/Engineering as OOD transfers). Three self‑evolution methods are compared: ACE (prompt‑visible skillbook accumulation), TF‑GRPO (grouped rollout evidence for updating experience/context store), and AHE (direct editing of the full harness).

Result 1 – Validation ≠ Stable Generalization : AHE improves validation from 40.0 to 57.1 (+17.1 pp), ID from 40.0 to 49.1 (+9.1 pp), and OOD from 22.5 to 28.8 (+6.3 pp). ACE shows modest gains (≈+3 pp across views). TF‑GRPO raises validation by +17.1 pp but drops OOD by 2.5 pp, illustrating that strong in‑distribution gains can hurt out‑of‑distribution robustness.

Result 2 – Intermediate Collapse : Replay diagnostics reveal that self‑evolution is not monotonic. In AHE’s train‑replay experiment, the agent solves 34/80 tasks initially and 43/80 finally, yet after epoch 4 performance falls to 6/80 with many rollout errors. The cause is a middleware/runtime contract change that breaks message construction; later updates fix the path, restoring performance. Thus, forgetting can manifest as tool‑path or contract failures rather than pure knowledge loss.

Result 3 – Batch Size Effects : Varying batch size (10, 20, 40, 80) shows a non‑monotonic relationship. Batch 10 degrades validation to 22.9 and ID to 23.6; batch 20 yields the best balance (validation 57.1, ID 49.1); batch 40 improves modestly; batch 80 collapses validation to 25.7 and ID to 25.5. Small batches provide insufficient evidence, while large batches dilute per‑task attention, both increasing runtime regression risk.

Result 4 – Training Source Diversity : Mixed‑source training (Terminal‑Bench + HLE) versus HLE‑only training shows that HLE‑only can achieve intermediate gains but ends with final validation, ID, and OOD scores of zero, indicating a collapse. Mixed sources supply diverse signals (tool usage, environment interaction, reasoning) that help the agent recover from bad states.

Result 5 – Backend Dependency : Cross‑model transfer experiments with DeepSeek, GLM, and GPT‑5.4 backends reveal that harness updates are not model‑agnostic. Same‑backend transfers improve ID scores (+9.1, +3.6, +5.5 respectively), whereas cross‑backend transfers are often asymmetric or negative (e.g., DeepSeek‑evolved harness gives GLM +7.3 but GPT‑5.4 ‑3.6). OOD gains are unstable, sometimes zero or negative, indicating that failure surfaces differ across backends.

Overall, SEAGym reframes self‑evolving agents as a reinforcement‑learning‑style evaluation pipeline, providing a unified framework and protocol to measure not only final success rates but also which harness components change, whether improvements generalize, whether old abilities are forgotten, and what costs are incurred.

Future work includes extending SEAGym to web and desktop interaction, long‑horizon software engineering, data‑analysis workflows, multi‑agent collaboration, and continuous online task streams, as well as developing more efficient snapshot selection, adaptive replay, and budget‑aware evaluation to mitigate the high token and time cost of multi‑view assessment.