From Harness to Environment: A Survey of Agentic Environment Engineering

The author introduces Agentic Environment Engineering as the next paradigm after "Harness" engineering, arguing that agents should interact with a dynamic environment through a closed loop of Observation → Action → State Update → Reward, turning fixed knowledge boundaries into a growth engine.

1. From Data Engineering to Environment Engineering

Figure 4 contrasts data engineering, where agents are passive learners of pre‑collected trajectories, with environment engineering, where agents co‑evolve with the environment via the closed‑loop interaction.

Core insight: the environment converts static knowledge limits into a dynamic capability growth engine, allowing data distribution to adapt to the agent’s abilities.

2. Formal Definition – POMDP Framework

The surveyed paper formalizes an Agentic Environment as a partially observable Markov decision process (POMDP) with the following components:

State space – the set of all potential environment states.

Action space – the set of actions an LLM can generate (text tokens or tool calls).

Transition function – the probabilistic kernel governing state changes.

Reward function – scalar feedback signals.

Observation space – the agent’s perception interface.

Observation function – mapping from state to observation.

Discount factor – weight of future rewards.

3. Eight‑Dimensional Attribute Taxonomy

Figure 5 enumerates eight core dimensions that characterize environments:

Symbolic vs Neural – rule‑based code engines (e.g., PDDL, Python) versus neural world‑models.

Open‑Loop vs Closed‑Loop – fixed plans based on initial observation (e.g., HuggingGPT) versus step‑wise observation‑driven adjustment (e.g., WebArena, MCPVerse).

Online vs Offline – real‑time interaction with dynamic systems (e.g., WebArena, Terminal‑Bench) versus static evaluation on pre‑sampled trajectories (e.g., Mind2Web, ALFRED).

MDP vs POMDP – fully observable settings (e.g., KOR‑Bench) versus partially observable ones (e.g., WebArena only sees the current page).

Deterministic vs Nondeterministic – fixed action outcomes (e.g., Baba Is AI) versus stochastic transitions (e.g., Frozen Lake).

Discrete vs Continuous – finite action sets (e.g., ALFWorld text commands) versus real‑valued vectors (e.g., RoboFactory joint control).

Unimodal vs Multimodal – pure text interfaces (e.g., API‑Bank) versus text + image + video (e.g., VisualWebArena, AgentStudio).

Single‑Agent vs Multi‑Agent – independent decision making (e.g., SWE‑Bench) versus joint action spaces (e.g., Collab‑Overcooked, AvalonBench).

Takeaway 3: current environments lack robust multi‑agent support; future work must balance symbolic reliability with neural scalability.

4. Eight Task‑Category Taxonomy

Figure 6 maps environments to eight task domains, each with representative benchmarks:

GUI – Desktop (OSWorld, WindowsAgentArena), Mobile (AitW, AndroidWorld), Web (WebShop, Mind2Web, WebArena).

Deep Research – Information Search (SimpleQA, WideSearch), Multi‑Source Reasoning (GAIA, BrowseComp), Report Writing (DeepResearch Bench, DR.BENCH).

Embodied – Spatial Navigation (Habitat, MetaDrive), Physical Manipulation (RLBench, Robocasa), Long‑Horizon Planning (ALFRED, TEACh).

Game – Open World (MineDojo), Puzzle Reasoning (Baba Is AI), Social Deduction (AvalonBench, Werewolf), Adventure Quest (FlashAdventure, BALROG), Strategy Management (CivRealm, Factorio Learning Environment).

Tool – Conventional (API‑Bank, ToolBench), User‑Simulated (τ‑bench, UserBench), MCP‑based (MCPVerse, MCP‑Bench).

Code – Generation (MBPP, BigCodeBench), Understanding (NL2Repo‑bench), Verification (LiveCodeBench), Debugging (InterCode, SWE‑Bench).

Domain‑Specific – Biomedical (MedAgentBench), Science & Technology (DiscoveryWorld), Finance (StockBench).

Cross‑Domain – Generalization benchmarks (OpenAI Gym, AgentBench, GEM, AutoEnv).

Key trend: a shift from static demonstration data toward executable, reproducible real‑world interaction.

5. Environment Synthesis

5.1 Symbolic Synthesis

Figure 7 shows three paradigms:

Task‑Driven – low freedom, static task data (e.g., SWE‑Gym, AgentScaler).

Real‑World‑Driven – medium freedom, mapping to real systems (e.g., AgentSynth, OSWorld‑MCP).

De Novo – high freedom, zero‑sample generation (e.g., AutoEnv, LOGIGEN).

Key evolution: synthesis freedom expands from task‑driven code wrappers to fully generative, minimal‑prior environments.

5.2 Neural Synthesis

Figure 8 categorizes neural synthesis by representation level:

Pixel‑Level – raw visual observations; high fidelity but computationally heavy (Matrix‑Game, NeuralOS, DIAMOND).

Word‑Level – natural‑language descriptions; abstract and lightweight but prone to information loss (WebDreamer, WKM, Code2World).

Latent‑Level – learned latent spaces; compact and predictive yet less interpretable (V‑JEPA 2, DINO‑world, seq‑JEPA).

5.3 Quality Evaluation Framework

Figure 9 proposes a four‑dimensional evaluation:

Correctness – does the transition work and is the task solvable? (program execution, unit tests, expert review).

Diversity – coverage of tasks, states, tools? (embedding deduplication, clustering, t‑SNE).

Complexity – is difficulty matched to agent capability? (step count, tool count, strong‑model win‑rate).

Fidelity – does the environment faithfully reflect the real system? (FID/FVD/LPIPS, Web Turing Score).

Takeaway 5.3: evaluation is moving from post‑generation filtering to a closed‑loop generate‑validate‑refine cycle; correctness is mature, while diversity, complexity, and fidelity are still nascent.

6. Four Complementary Paths for Agent Evolution

6.1 Memory‑Centric Experience Evolution

Instance Trajectory – full interaction traces (Synapse, WorldMM, OpenAgent).

Abstract Scripts – reusable script templates (Reasoning‑Bank, Agent‑Pro, Agent‑KB).

Structured Skill – modular skill libraries (SAGE, SkillWeaver, SkillRL, SkillOrchestra).

6.2 Orchestration‑Centric Workflow Evolution

Fixed Workflow – deterministic graphs pre‑specified (MetaGPT, Agentless, MedAgent‑Zero).

Automated Workflow – dynamic graphs coordinated by a central controller (AutoFlow, MaAS, Workforce, Mindsearch).

Evolving Workflow – self‑iterating structures (AFlow, AgentSquare, Chain‑of‑Agents, LATM).

6.3 Trajectory‑Centric Offline Evolution

Three‑stage pipeline:

Task Synthesis – resource conversion, reverse engineering, structural synthesis.

Trajectory Synthesis – augmentation, sequential interaction, tree search, model simulation.

Trajectory Refinement – filtering, correction, iterative optimization.

6.4 Exploration‑Centric Online Evolution

Reasoning Structure – modify prompting format/steps (Search‑R1, AutoRefine, Video‑Thinker).

Training Reward – multi‑dimensional signals (ToolRL, GDPO, IntentRL, SHARP).

Algorithm Optimization – stability and sample efficiency (RAGEN, GiGPO, DigiRL, ComputerRL).

7. Three Paradigms for Environment Evolution

7.1 Neural‑Driven Evolution

Self‑Play – agents act as proposer, solver, challenger (Absolute Zero, Self‑Challenging, Vision‑Zero).

World Model – learn simulators that approximate dynamics (WebDreamer, Code2World, WebWorld).

7.2 Difficulty‑Driven Evolution

Explicit Curriculum – signals based on accuracy, regret, curiosity (RLVE, SCALER, PAIRED, ACCEL).

Implicit Curriculum – emergent difficulty from task generation (POET, DreamGym, AgentGen, Reasoning Core).

7.3 Scaling‑Driven Evolution

Scenario‑Level – increase diversity of tasks, trajectories, websites within a paradigm (AgentScaler, EnvScaler, InfiniteWeb).

Environment‑Level – cross‑domain expansion of environment structure (ARE, AutoEnv).

Final insight: Agentic Environment Engineering is more than building playgrounds; it constructs an evolvable, verifiable, and scalable cognitive infrastructure that underpins the shift from training models to cultivating intelligent agents.

https://arxiv.org/pdf/2606.12191
Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application