Skill2Env: Auto-Generating Evolving Agent Training Tasks from Skills

From Skills to Executable Training Environments

As agents move from answering questions to executing real work, training must shift to interactive environments where models explore, plan, call tools, handle errors, and adjust strategies based on feedback. A key challenge is scaling the construction of environments that are both truly executable and sufficiently challenging. Existing skill repositories (e.g., ClawHub) contain rich domain knowledge, tool usage procedures, and operational experience, but converting them into effective training environments requires answering: which capabilities to exercise, how to translate those capabilities into concrete task challenges grounded in an environment, and how to evolve the environment as the agent masters current tasks.

The AllSpark team downloaded over 47,000 skill folders from ClawHub, filtering for real-world applicability, sandbox compatibility, and feasible dependency configuration, retaining roughly 3,000 skills across 25 domains. These skills provide the raw material—diverse knowledge, tools, and artifacts—for constructing varied tasks.

Capability-Driven Difficulty Patterns and Task Blueprints

Skill2Env distills complex task execution into five core capability dimensions:

Environment Understanding : Identify relevant facts from the environment, grasp task goals and constraints.

Planning : Formulate multi-step execution plans and dynamically adjust them during execution.

Skill Usage : Correctly interpret and apply the tool descriptions and workflows provided by a skill.

Long-Horizon Consistency : Track state and context over long tasks to maintain execution coherence.

Error Recovery : Diagnose issues from execution feedback, adjust actions, and continue.

To operationalize these capabilities, the team designed a reusable difficulty pattern pool . Examples include "scattered evidence" (key facts distributed across sources, requiring synthesis) and "state dependency" (later steps depend on earlier results, enforcing ordering). A task synthesis agent selects compatible patterns for a given skill's workflow and produces a task blueprint that specifies the task goal, expected challenges, environment facts, acceptance criteria, and information boundaries (what is given in the instruction, what must be discovered in the workspace, what must be inferred).

Around each blueprint, the system co-constructs the task instruction, runtime dependencies, initial workspace, and an evaluator. Programmatically verifiable requirements use deterministic checks; others use an LLM judge. Scores are weighted and aggregated with supporting evidence logged. After each execution, the system cross-checks environment–evaluator consistency to fix construction issues (missing materials, scoring errors) so they are not mistaken for agent inability.

Iterative Task Strengthening Driven by Agent Performance

Initial challenges may be too easy or circumvented by shortcuts. Iterative task strengthening examines execution trajectories and scores: tasks where the agent's reward exceeds a threshold enter the next strengthening round; others are retained. A diagnostic agent analyzes which challenges were solved, which requirements were bypassed, and what problems remain. The system then reinforces existing difficulty patterns or introduces new compatible ones, updating the blueprint first, then synchronizing the instruction, workspace, environment, and evaluator. Previously uncovered generic challenges can be abstracted into new difficulty patterns. Thus, difficulty escalation is driven by the agent's actual performance, not preset complexity.

Case Study: Docling Skill to Multi-Round Vendor Reconciliation

The paper illustrates the full pipeline with the Docling skill (document parsing, format conversion, table extraction, OCR). The initial task: reconcile a supplier's quarterly statement using 10 files in 6 formats (DOCX contract, HTML rate annex, PDF invoices, scanned receipts, XLSX payment ledger, Markdown notes) and produce three consistent deliverables (CSV detail, JSON summary, audit notes). The task required judging rate-notice effectiveness, verifying travel receipts, excluding unrelated historical materials, and ensuring cross-deliverable numerical consistency. Three designed reconciliation anomalies were all caught on the first run (perfect score).

First strengthening round extended the workflow: key rates moved into a signed addendum, cross-month duplicate reimbursements and distractor materials from another project were added, invoice arithmetic errors and partial payments introduced. The agent now had to distinguish original billed amounts, corrected amounts, and actual payments.

Analysis of the second execution revealed OCR conversions aborted repeatedly due to command-parameter issues; the agent fell back to reading images directly for receipt amounts. Second strengthening round therefore placed critical evidence in scanned PDFs without text layers, forcing reliable conversion and extraction. It also added date-dependent rates, unsubstantiated claims, and cross-tier day-cap deductions.

After two rounds, workspace files grew from 10 to 21 , designed anomalies from 3 to 12 , and anomalies became interdependent—extraction, scope judgment, rate calculation, and settlement verification had to form a coherent pipeline to yield consistent results.

Experimental Results: 1.5K Trajectories Boost Performance Across Benchmarks

Skill2Env produced 2,963 executable tasks across 25 domains, with initial workspaces containing 46,295 files (code, documents, structured data, images). Using DeepSeek-V4-Flash, the team generated trajectories in these environments and selected 1,500 high-scoring trajectories to supervised-fine-tune Qwen3.6-35B-A3B.

Crucially, no training environment was tailored to any downstream benchmark. Yet the fine-tuned model improved on all seven evaluated benchmarks, covering terminal operation, code repair, skill usage, autonomous task execution, banking support, cross-application workflows, and interactive service tasks—demonstrating broad generalization.

Cross-Framework and Cross-Domain Generalization

On Terminal-Bench 2.1, the team swapped the execution harness (tool-call/interaction framework) across four systems: pi, Terminus-2, OpenCode, and Claude Code. The fine-tuned model gained 11.2, 13.5, 16.9, and 12.3 percentage points respectively, confirming framework-agnostic gains.

On SkillsBench (eight domains), with automatic skill loading enabled ( Skills on ), seven domains improved; finance & economics was flat. Media & content, mathematics & operations research, and software engineering rose by 46.7, 26.9, and 23.4 percentage points . With skill loading disabled ( Skills off , though skill files remained accessible), all eight domains improved.

Conclusion and Future Work

Skill2Env presents a capability-oriented framework for synthesizing agent training environments from existing skills. By leveraging reusable difficulty patterns and task blueprints, it translates capability requirements into concrete, executable, evaluable challenges and uses execution feedback to continuously harden tasks. Experiments show the resulting training data lifts agent performance across diverse benchmarks. As agents grow stronger, environments must offer more targeted, challenging tasks. Future work includes accurately pinpointing model capability gaps and dynamically adapting task design so environments co-evolve with agent abilities.

Paper: Skill2Env: Capability-Oriented Environment Synthesis from Skills for General Agents (arXiv:2609.33772) Code:

https://github.com/AllSpark-Research/AgentEnv/tree/main