Beyond Model Scaling: How Agent Training Shifts from Bulk Environments to Designed Worlds

From Model Scaling to Environment Scaling

For the past few years, large‑model capability grew along a clear path: more parameters, more data, more compute. Once agents entered the scene, simply enlarging the model proved insufficient because an agent must experience diverse tools, state changes, failure feedback, and long‑chain tasks to act reliably in unfamiliar systems.

Dataset Scaling → Experience Scaling

Traditional language models train on static samples: input → prediction → loss. Agents, however, operate on interactive loops where each tool call can modify backend state and the new state determines subsequent actions. For example, an e‑commerce agent that must cancel an already‑received order and a pending order must first query the order status, decide which tool to invoke, and handle the resulting state transition. Thus agents need to learn from sequences of Action, State Transition, Observation, Correction rather than isolated question‑answer pairs.

EnvScaler: Programmatic Synthesis of Tool‑Interactive Environments

EnvScaler tackles the engineering bottleneck of obtaining environments. It consists of SkelBuilder (defines environment theme, state, tools, and constraints) and ScenGenerator (initializes database state, creates tasks, and generates rule‑based trajectory validators). The system retained 191 distinct environments, yielding roughly 7,000 task scenarios. Each environment averages 18.58 tools, 21.38 state categories, and 4.58 constraint rules.

When these environments were used to train Qwen3‑8B models, performance improved markedly: BFCL‑v3 Multi‑Turn Overall rose from 28.88 to 37.00 (SFT) and to 41.88 after RL; Tau‑Bench increased from 38.19 to 41.35 and then to 44.81; ACEBench‑Agent grew from 60.00 to 71.67 and finally to 72.50. Adding environments from 0 to 10, 20, 50, and 141 showed continuous gains, with the steepest improvement between 0 and 20 environments and diminishing returns thereafter.

AgentScaler: Scaling Across Heterogeneous Tool Worlds

AgentScaler extends EnvScaler by collecting >30 k APIs (ToolBench, API‑Gen, etc.), constructing a Tool Dependency Graph, and organizing >1 000 domains. Function calls are modeled as real read/write operations on the underlying environment database, producing genuine state transitions.

Experiments varying the number of training environments (0, 1 000, 2 000, 4 000, 10 000) showed ACEBench‑en scores of 67.2 → 70.2 → 70.5 → 72.6 → 75.7, which the authors term a “Scaling Law of Environments.” While not a formal mathematical law, the trend indicates that expanding environment coverage continues to boost agent ability when the base model is fixed.

Echoverse: Deep, Evolving Environments

Echoverse identifies three high‑value environment attributes: Behavioural Depth , Failure Targeting , and Co‑evolution . Experiments comparing shallow and deep environments within the same domain showed that shallow environments reduced a real‑website accuracy metric from 80.0 % to 75.0 %, whereas deep environments raised it to 85.0 %. Another metric improved from 48.0 % to 65.0 % with depth.

Repairing a single failing environment lifted the model’s success rate on that environment from 16.2 % to 38.5 %. Training a 9 B model on 12 environments increased overall performance across 14 evaluation splits from 36.5 % to 67.1 %; further RL training raised held‑out scores from 58.8 % to 68.0 %.

Beyond Simply Environment Scaling: Designing Effective Distributions

The latest research shifts focus from “how many environments” to “what environment distribution.” Effective distribution is broken into Diversity (covering distinct agent capability regions) and Difficulty Structure** (curriculum‑like progression). The paper proposes Ability‑aware Environment Selection (AES) to pick environments matched to the agent’s current ability, and Hierarchical Difficulty Curriculum (HDC) that gradually increases difficulty via harness weakening and state‑scale progression.

Synthesis: From Quantity to Quality of Experience

Putting EnvScaler, AgentScaler, Echoverse, and the August 4th study together reveals a clear evolution: (1) scale environment quantity; (2) improve environment diversity; (3) deepen depth, difficulty, failure coverage, and co‑evolution. The true scaling axis is the Experience Space , not merely task count or tool count.

Consequently, the training pipeline for agents must evolve from a traditional dataset pipeline (collection → cleaning → tokenization → training) to an Experience Pipeline that adds Environment Construction, State Initialization, Task Generation, Agent Rollout, State Transition, Trajectory Verification, Reward/Filtering, and later Failure Analysis, Environment Selection, Difficulty Adjustment, and Environment Repair.

Future Outlook

By August 2026, the consensus is that environment scaling is becoming a distinct capability axis. The next frontier is building a continuous “Environment Factory” that constantly discovers an agent’s ability gaps, generates new states, rules, tasks, and failure scenarios, and lets the agent co‑evolve with its worlds. Ultimately, an agent’s ceiling will depend not only on model size but also on the richness and distribution of the worlds it has trained in.

References: Song et al., 2026 (EnvScaler); Fang et al., 2026 (AgentScaler); Zhang et al., 2026 (Scalable Environments); Pandya et al., 2026 (Echoverse); Zhu et al., 2026 (Beyond Simply Environment Scaling).