NVIDIA's Skill2Env and SoL-Pi: Collective Skills to Self-Evolving Agent Harnesses

Skill2Env: Compiling Collective Skills into Training Environments

Traditional terminal-agent training data comes from code repositories or human-designed tasks, biasing toward software engineering. Skill2Env instead ingests publicly shared skill packages from practitioners and compiles them into executable, verifiable terminal environments.

Four-Stage Pipeline

Planning : Codex reads the full skill, retrieves public materials, and extracts verifiable workflows.

Diversification : Samples across task type, verification method, complexity, role, and expertise level to avoid homogeneous tasks.

Creation : Generates initial environment, task instructions, programmatic tests, behavioral rubrics, and reference solutions.

Acceptance : Reference solutions must pass all tests; null actions must fail; safety checks and base-image pinning are enforced.

Tests and rubrics are frozen before reference solutions, preventing the acceptance criteria from being rewritten to fit the solution. Training signals are split into outcome correctness and process quality, so agents learn not only to complete tasks but also how experts judge quality.

Approximately 3,400 skills were compiled into 7,971 tasks covering 13 domains; software engineering accounts for only 22.5%, shifting the training distribution toward real-world knowledge work.

Results

After 150 steps of FlashREINFORCE, Qwen3.8-27B improves on Terminal-Bench 2.1 from 49.4 to 58.4 and on the private S2EBench from 33.4 to 40.5.

Both DPPO and FlashREINFORCE raise scores; FlashREINFORCE adds 9.0 and 7.1 percentage points on the two benchmarks, and S2EBench average rises by 21.9 points.

Behavioral Alignment

Beyond pass rates, the authors encode skill methods as behavioral rubrics and have an LLM review trajectories. In 200 paired comparisons, optimizing only for outcomes yields a 54.5% preference for RL trajectories; adding rubrics lifts this to 73.0%.

The key contribution is not merely training another model but turning scattered community practices into repeatable training infrastructure.

SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness

While Skill2Env addresses training content, SoL-Pi tackles token and API costs during long-horizon task execution. It does not train a new model nor allow unrestricted self-rewriting; instead it optimizes the Pi harness logic responsible for tool calls, context management, observation handling, and task delegation.

An AI researcher reads execution traces left by another agent on the base harness, proposes candidate modifications, and filters them through capability and efficiency thresholds. The search spans ~152 directions, 535 executable environments, and over 3,000 runs, ultimately retaining four mechanisms:

Action Fusion : Merges file edits with subsequent commands to reduce model round-trips.

Online Context Compact : Evaluates compression gain at sub-task completion to avoid wasteful cache rewrites.

ObservationPack : Stores long outputs in full initially, then replaces them with handles and summaries, retrieving originals only when needed.

Evidence-Preserving Reducer : Uses a small model to distill build and test logs, followed by a deterministic verifier that checks the evidence.

Crucially, holdout evaluation runs only after candidates are frozen, and its results never flow back into the search, isolating development feedback from final verification to reduce overfitting risk.

This design is not open-ended self-evolution but a verifiable, freezable, rollback-capable engineering process for harness improvement.

Results

On GPT-5.6 Sol, SoL-Pi reduces token traffic by 49.0% and API cost by 33.2% while retaining 93.7% of the average score. When transferred to Opus 5 without re-searching, it still cuts token traffic by 44.7% and cost by 33.5%.

Efficiency gains come with a trade-off: on Terminal-Bench 4, SoL-Pi solves 15 tasks versus 18 for the base Pi, confirming a real capability ceiling reduction.

NVIDIA, Skill2Env: Reinforcing Agents with Collective Skills,
arXiv: 2609.33772
NVIDIA, NTU, MIT, SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness,
arXiv: 2609.20519