How the [schema] Harness Achieved 99% RHAE on ARC‑AGI‑3 by Making AI Think Like a Physicist

On July 16 a tweet by Berkeley post‑doc Haven Feng went viral because a combination of the [schema] harness with Claude Opus 4.8 and Fable 5 reached a 98.98% RHAE score on the public ARC‑AGI‑3 set; the same harness paired with GPT‑5.6 Sol still scored 95.35%.

ARC‑AGI‑3, released in March, originally saw the strongest agents achieve only 0.51% on the public set. By July the best officially verified result was 7.78% (semi‑private) for GPT‑5.6 Sol, and 13.33% on the public set. The [schema] harness therefore represents a dramatic leap.

The key insight is that the improvement does not come from changing model weights but from adding an external “harness” layer. This layer forces the model to operate in a four‑step loop—observe, hypothesize, execute, record—turning its world understanding into an explicit, runnable program that can be back‑tested and searched.

Four‑step loop : The agent observes the game grid, proposes a hypothesis about the underlying mechanism, executes an action, and records the outcome. The recorded history is immutable; the agent may revise its hypothesis or state representation but cannot alter past observations.

State grounding and mechanism discovery : The harness splits the problem into (1) grounding raw pixels into traceable objects, variables, and relations, and (2) discovering how those states change under an action and encoding the rule as executable code. Both steps are solved jointly: if a prediction contradicts an observation, the agent can edit either the state definition or the transition rule, keeping the model self‑consistent.

To illustrate, the authors compare the process to the historical development of special relativity: Lorentz patched the ether hypothesis, whereas Einstein removed the ether from the state definition, leading to a cleaner theory. Similarly, the harness lets the AI revise what “state” means when hypotheses fail.

By encoding the world model as code, the harness gains three properties:

Interpretability : the program is a readable text file.

Verifiability : it can be replayed against the full interaction history to check each belief.

Searchability : the program acts as a simulator, allowing planning without real environment steps.

When multiple candidate rules explain the history, the agent designs a discriminating experiment—an action whose predictions differ across rules—so that a single real interaction can resolve the ambiguity. Because the RHAE metric penalizes unnecessary actions quadratically, such targeted experiments dramatically reduce the total number of steps.

Empirical results on 25 public games show that in 14 games the agent produced a precise program world model, cutting the number of actions needed by a factor of 1.6 to 5 compared with human baselines. In an extreme case (M0R0 level 4) the agent used 42 actions versus 500 for a human, not because it knew the solution a priori but because it reused mechanisms discovered in earlier levels.

Comparisons highlight the harness’s impact: the same Claude Opus 4.8 + Fable 5 baseline achieved only 42.83% RHAE, while adding the [schema] harness raised it to 98.98%, a 56.15‑point jump with the model unchanged. The authors note that the 13.33% → 95.35% gap for the Sol max baseline is not a fair harness‑vs‑harness comparison because the former was measured under a minimal‑shell configuration.

Critics raised concerns about “cheating.” Greg Kamradt, ARC Prize president, pointed out that the reported scores rely on a fixed fallback rule that switches models based on per‑level performance, effectively injecting human knowledge into the process. He also noted that the official documentation mentions concepts like “player” and “wall,” suggesting that human‑provided hints may inflate scores.

Kamradt further emphasized that the 98.98% and 95.35% figures are self‑reported on the public set and have not been independently audited by ARC Prize; the [schema] team acknowledges this in their blog.

Finally, the authors caution that the public ARC‑AGI‑3 set is easier than the semi‑private set, and a near‑perfect public score does not guarantee similar performance on harder benchmarks. Nonetheless, the core contribution—forcing a model to express its world understanding as an executable, falsifiable program and to iteratively test hypotheses—embodies the scientific method and opens a path toward general mechanism‑discovery capabilities beyond the 64×64 grid.