MEMENTO 3: Frozen LLM Agents Evolve World Models via Code-as-Model

Overview

Huawei Noah's Ark Lab, in collaboration with University College London and the University of Liverpool, presents MEMENTO 3 — an AI agent system that achieves recursive self-improvement (RSI) without retraining the underlying large language model. Instead of updating model parameters, MEMENTO 3 freezes the base LLM (Claude Opus 5, reasoning set to extra-high) and equips the agent with an external, continuously updated world model composed of a natural-language rulebook and executable code. The agent interacts with an environment, observes discrepancies between its predictions and reality, revises its rules, regenerates code, and validates the new model against all past observations before using it for further planning.

Code as Model: Rulebook and Executable World Model

The core design, Code as Model , splits the agent's understanding into two parts:

A Rulebook written in natural language that records the agent's current hypotheses about environment mechanics.

An executable world model (Python code) generated from the rulebook to simulate state transitions, predict action outcomes, and plan routes.

This gives the agent a modifiable "game guide" and a "game engine" for mental simulation. When facing a novel situation, the agent can first reason inside its own world model before committing real actions.

Five-Step Recursive Self-Improvement Loop

MEMENTO 3 runs a continuous learning cycle:

Observe — Execute an action, record before/after frames, store in interaction history.

Reflect — Compare actual observation with the world model's prediction.

Revise Rules — If mismatch, diagnose whether the rulebook or the code implementation is at fault; update the rulebook or fix the code accordingly.

Compile Code — Regenerate the executable world model from the revised rulebook.

Verify — Double verification: (a) LLM checks code fidelity to rulebook; (b) system re-runs all recorded actions through the new code and demands pixel-perfect match (64×64 grid) with historical observations.

Only after passing both checks does the new model replace the old one. Failures trigger another reflection-revision cycle.

Double Verification Mechanism

The verification step is critical. In ARC-AGI-3 experiments, the agent's predicted frames had to match the true 64×64 grids exactly. This prevents hallucinated rules from persisting and ensures the world model remains grounded in actual experience.

Git-Versioned External Memory

The system uses Git to version-control every change to the rulebook and code. The agent can inspect history, compare versions, and roll back — turning external memory into an evolving knowledge base rather than a simple chat log.

ARC-AGI-3 Experiments

Tested on the 25 public ARC-AGI-3 games (183 levels total), MEMENTO 3 achieved:

100% completion — all 183 levels cleared.

Average RHAE (Relative Human Action Efficiency) = 100.0 (maximum score).

Total actions = 7,518 vs. human baseline 17,135 — only 44% of human actions .

For comparison, a baseline method (baseline1) also cleared all games but used 8,347 actions and scored RHAE 99.0.

Ablation Study: Rulebook Impact

Removing the natural-language rulebook (keeping executable model and other settings) on three games of varying difficulty increased total actions from 617 to 677 (+9%) and internal iterations from 678 to 830 (+18%). The explicit rulebook helps the agent correct its model faster, reducing exploration cost.

Atari Pong: From Planning to Real-Time Control

MEMENTO 3 was also applied to the classic Atari game Pong, which requires real-time paddle control based on continuous ball dynamics. The agent learned ball physics, bounce rules, and opponent behavior, writing them into its rulebook and code. The resulting controller:

Won three test games 21–0 (different initial conditions).

In the final test phase, operated without any LLM calls — using only the learned executable world model to predict action outcomes and drive the paddle.

Scored +21 vs. −19 for an otherwise identical agent lacking a world model.

This demonstrates that the acquired environment rules can be distilled into a standalone, LLM-free controller.

Illustrative Example: Hypothesis Revision

The article describes a concrete scenario: the agent encounters a mechanism that rotates a pattern on the avatar. Initially it hypothesizes a left-right flip. After entering from another direction and seeing the same rotation direction, it revises the rule to "clockwise 90° rotation regardless of entry direction." This exemplifies the observe-reflect-revise cycle at the heart of MEMENTO 3.

Key Takeaways

Recursive self-improvement can be achieved by freezing the base LLM and evolving an external, code-based world model.

The Code as Model paradigm (rulebook + executable code) enables explicit, verifiable, and versionable knowledge.

Double verification (LLM code review + empirical replay) keeps the world model grounded.

MEMENTO 3 outperforms strong baselines on ARC-AGI-3 and transfers learned models to real-time control without LLM inference.