AI Agents Evolve by Reading Research: Microsoft & UCSB's ScholarEvolve

Introduction

As large language models advance, AI agents built around them increasingly handle complex, multi-step tasks. However, agent performance depends not only on the backbone model but also on the surrounding software system—called the Agent Harness —which manages tool calling, context, skills, memory, and workflow planning.

Problem: Limited Self-Improvement

Existing approaches like Meta Harness use a coding agent to analyze failures and modify harness code. This relies heavily on the model's pre-existing knowledge and observed failures; it cannot proactively explore improvements for problems that haven't yet appeared or methods the model doesn't know.

ScholarEvolve: Automating the Research Loop

ScholarEvolve automates the full research cycle: reading papers, implementing algorithms, evaluating them, and merging verified improvements into the harness. It decomposes the harness into five modules:

Tools – interface definitions and calling logic

Context – context window management and compression

Skills – reusable procedural knowledge

Memory – long-term storage and retrieval

Workflow – task planning and execution orchestration

Step 1: Problem-Driven Literature Retrieval

The system analyzes execution traces to identify recurring issues, translates them into capability needs, and builds retrieval queries per module. It fetches candidate papers from arXiv.

Step 2: Diverse Exploration via Topic Modeling

Instead of picking only the most relevant papers (which often share similar methods), ScholarEvolve applies topic modeling to cluster retrieved papers by technical mechanism . For example, context management papers may use compression or retrieval—different mechanisms solving similar problems. This ensures diverse coverage under a limited experiment budget.

Step 3: Implementation, Testing, and Composition

A Research Agent reads selected papers (including GitHub code and appendix prompts) and produces implementation plans. A Coding Agent modifies the corresponding harness module. Each candidate is tested on validation tasks. Since a module that works in isolation may not compose well, ScholarEvolve also searches combinations of module improvements and retains only those that improve the full harness.

Experimental Setup

Benchmarks: AppWorld (API-driven multi-app tasks, with Normal and Challenge splits; Challenge includes unseen apps like Amazon and Gmail) and τ²-Bench Telecom (tool-calling with user interaction). Backbone models: Qwen3.5-27B and GPT-5.4-mini . Baselines: original harness and Meta Harness.

Results

AppWorld Normal (Qwen3.5-27B) : task completion rate 69.0% → 81.4% .

AppWorld Challenge (Qwen3.5-27B) : 49.6% → 63.6% (Meta Harness: 54.6%).

AppWorld Challenge (GPT-5.4-mini) : 46.1% → 55.6% .

τ²-Bench Telecom (GPT-5.4-mini) : single-task success rate 72.7% → 81.9% .

All gains come from harness evolution only; backbone model parameters remain frozen.

Ablation Studies

Removing literature guidance, modular search, or topic modeling each degrades performance, confirming that improvement stems from how methods are discovered, organized, and combined—not just from automated code editing. Notably, the best-performing individual modules do not necessarily yield the best combined harness, highlighting inter-module dependencies.

Lifelong Evolution Simulation

To test continuous improvement, the team created three sequential research windows based on paper publication dates (Dec 2025, Jan–Apr 2026, May–Aug 2026). With Qwen3.5-27B on AppWorld Normal, performance climbed stepwise: 69.0% → 73.4% → 79.8% → 81.5% , demonstrating that new papers can drive further gains even without new failures.

Limitations and Outlook

Evolved harnesses still struggle with constraint tracking and information retrieval completeness in complex tasks. Some paper-derived implementations hurt performance, so empirical evaluation remains essential. ScholarEvolve bridges the gap between publication and practical integration, pointing toward agents that continuously learn from the growing research literature.

Paper: Learning from Research: Toward Lifelong Agent Harness Evolution (arXiv:2609.40169). Code: https://github.com/UCSB-NLP-Chang/ScholarEvolve. Lead author: Jingbo Yang (UCSB PhD, Microsoft Research intern).