How Peking University’s Two Papers Redefine Agent Skill Evolution

SESA: Self‑Play Meets Skill Evolution

SESA addresses tool‑augmented search‑QA agents. Self‑play generates questions and answers but only provides a gradient from failures, leaving no persistent state. An external Skill memory learns from a fixed task distribution, remembering but unable to select questions. The two shortcomings are complementary: self‑play learns but forgets; Skill memory remembers but cannot pose challenges.

SESA closes the loop in four stages:

Memory initialization : 15 hand‑written skills plus 142 bootstrap‑extracted skills, for a total of 157 base skills.

Asymmetric self‑play : a challenger poses questions, a solver answers; only the solver can retrieve skills, preventing the challenger from learning the solver’s strategy.

Frontier shaping : a bell‑shaped reward pushes the challenger toward problems near the solver’s ability boundary, penalizing overly easy or overly hard questions so that failures remain learnable.

Failure distillation : useful failures are distilled into new skills, deduplicated, filtered by a help‑hurt criterion, and capped at 800 entries.

The loop relies on bidirectional coupling: updated skills change solver behavior, a stronger solver alters challenger rewards and the distribution of future questions, which in turn creates new failures that rewrite the memory.

Key empirical findings:

Disabling memory during inference (SESA‑Off) still yields a solver that outperforms the SSP baseline by +1.8 points (Qwen3‑4B) and +2.2 points (Qwen3‑8B), indicating that skill learning is baked into model parameters.

Re‑enabling memory adds an additional +0.5 to +1.0 points.

Across seven QA benchmarks, SESA improves over SSP by 1.2–3.2 points on average (7.0–11.8 absolute gain) for base models Qwen3, Qwen2.5, LLaMA‑3.1, and Search‑R1.

Ablation shows failure distillation contributes the most (+2.7 points), followed by frontier shaping (+2.2) and memory initialization (+1.5).

Compared with the published SkillRL, SESA scores 51.0 versus 50.1, a 0.9‑point advantage.

Training dynamics show verification quality rising steadily then stabilizing, while the number of active skills first expands and later contracts due to deduplication and pruning, demonstrating a self‑refining memory rather than unbounded accumulation.

VeriSkill: Turning Verification Failures into Trustworthy Skill Updates

Program verification uses tools such as Dafny, Frama‑C, and VeriFast to prove that code satisfies formal specifications. An LLM agent requires reusable skills to automate verification, but verifier feedback is often opaque.

VeriSkill converts noisy failure signals into reliable skill updates through three steps:

Responsibility attribution : each failure is classified into four categories—unsatisfiable task, verifier limitation, non‑compliant skill, or knowledge gap. The first two are discarded; the latter two become evolution signals, filtering out the largest source of noise.

Lesson abstraction : failures are clustered by pattern, and a reusable lesson is abstracted by stripping instance‑specific details, avoiding hard‑coded prompts.

Executable verification : candidate skills are repeatedly run on the attribution set and a pattern‑transfer set; only skills that strictly improve verification success without breaking program semantics are admitted to the skill library, and all results are written back to the evolving memory for the next round.

Results:

Across three verifiers and two agent configurations, VeriSkill achieves six‑fold wins. Relative to a no‑skill baseline, Dafny improves by +43.3 points, Frama‑C by +17.6 points, and VeriFast by +46.0 points (using Claude Code/Opus). With Codex/GPT, VeriFast gains +51.3 points.

Against the strongest baseline SkillOpt‑Lite, VeriSkill still leads by +15.0 (Dafny), +9.1 (Frama‑C), and +5.7 (VeriFast) points.

Ablation confirms each step is essential: removing executable verification drops PASS from 57.3 to 36.0 (‑21.3 points); removing responsibility attribution loses 13.0 points; removing lesson abstraction loses 8.3 points.

Transferability: skills evolved with Codex, when applied directly to other models within the Terminus framework, still boost performance by +15.3 to +43.7 points, showing that VeriSkill extracts reusable knowledge rather than model‑specific prompt tricks.

VeriSkill paper: https://arxiv.org/html/2607.27733v1
SESA paper: https://arxiv.org/html/2607.29468v1
GitHub repository: https://github.com/Zenghuang-Fu/SESA-Self-Evolving-Search-Agents