Tsinghua AI4S Breakthrough: Agents Discover Formulas and Mechanisms

Introduction: Two Steps of Scientific Discovery

The article frames scientific discovery as a two-step process: first, inducing mathematical laws from observational data (like Kepler's planetary motion laws), and second, explaining the underlying physical mechanisms (like Newton's gravity). Today, large language models (LLMs) and scientific agents can autonomously analyze data, call tools, and iterate hypotheses. The key questions are whether AI can reliably discover stable laws from data and further understand the mechanisms behind those laws.

Tsinghua University's Data Science and Intelligence Laboratory addresses both questions with two complementary contributions: SRHarness, a runtime framework for agentic symbolic regression that reliably discovers mathematical formulas, and MechBench, a benchmark that systematically evaluates whether AI can infer hidden mechanisms from limited observations.

SRHarness: Reliable Formula Discovery

Motivation

As symbolic regression evolves from LLM formula generation to autonomous agent-driven analysis, tool use, and iterative hypothesis testing, the supporting runtime infrastructure becomes critical. Existing implementations are often tied to specific methods and lack reusability.

Framework Design

SRHarness makes this infrastructure explicit and reusable, without prescribing a particular search strategy. It provides three core mechanisms (Figure 1):

Reusable Scientific Actions : Standardizes heterogeneous scientific operations (data analysis, numerical optimization, symbolic engines, candidate evaluation) behind unified interfaces with consistent evaluation semantics, letting agents focus on what to do rather than low-level execution details.

Persistent Scientific State : Continuously saves candidate formulas and their scientific evidence, making hypotheses, evaluation results, complexity, and provenance independent of transient conversation context, enabling cross-turn comparison, ranking, and reuse.

Trajectory Lifecycle Management : Manages continuation, branching, restart, and termination of long-horizon search, supporting traceability, failure recovery, and best-result retention, making long scientific searches observable, auditable, and recoverable.

SRHarness is released as a complete agent system covering data analysis, numerical fitting, symbolic search, candidate management, unified evaluation, state persistence, trajectory tracking, and failure recovery. Researchers can extend it with domain-specific tools and search strategies.

Open-Source Resources

GitHub: https://github.com/tsinghua-fib-lab/SRHarness Documentation: https://yuzhthu.github.io/SRHarness/ Online Demo:

http://sim1.fiblab.tech:30000/

Experimental Results

On the LSR-Transform benchmark with DeepSeek-v4-flash, SRHarness achieves a 93.69% symbolic recovery rate , surpassing the previous state-of-the-art SR-Scientist (ICLR 2026) at 62.16% — an absolute improvement of over 30 percentage points (~50% relative). On LSR-Synth across physics, chemistry, biology, and materials, out-of-distribution numerical prediction accuracy reaches 78.00%, 78.87%, 87.09%, and 96.00% respectively.

Crucially, even when scientific context and variable semantics are removed (LSR-Transform-Anon), SRHarness still attains 72.97% recovery , far above SR-Scientist (39.64%) and Codex with the same base model (20.72%), and even exceeds Codex with GPT-5.5 (68.47%). With the more economical DeepSeek-v4.1-flash, SRHarness reaches 95.52% in-distribution and 92.41% out-of-distribution overall accuracy on LSR-Synth, approaching Codex powered by GPT-6-Astra (biology 98.59% vs 98.59%, materials 100% vs 100%).

Ablation studies (Figure 2) confirm all three mechanisms are indispensable: removing key components of reusable actions, persistent state, or trajectory management each causes significant drops in symbolic recovery.

MechBench: Can AI Discover Mechanisms Beyond Phenomenological Laws?

Problem Definition

Discovering a mathematical formula does not equate to understanding the physical mechanism. Current symbolic regression methods and benchmarks target phenomenological models that merely fit observations. MechBench explicitly distinguishes recovering observational laws from recovering the generative mechanisms (Figure 3). Mechanistic models describe internal components, relations, and organization that produce the phenomena and entail internal scientific quantities not determined by the phenomenological law alone.

Benchmark Construction

MechBench comprises 16 real-world mechanism task families from physics, chemistry, biology, and materials. Scientifically meaningful controlled mutations are applied to familiar classic mechanisms to create unfamiliar systems, preventing models from simply reciting textbook answers. This yields 512 problems (plus an 80-problem core set for detailed analysis). Each problem provides only observational data and scientific context; the agent must recover both the observable law and the hidden mechanism.

Automated, objective verification uses two novel techniques (Figure 4):

Mechanism Probes : Test whether a submitted mechanism can correctly derive internal scientific quantities (e.g., centripetal acceleration) that the phenomenological law cannot determine.

Competitive Mechanism Screening : Identifies and excludes ambiguous instances where multiple scientifically plausible mechanisms produce identical observations (mechanistic indistinguishability).

Capability Gap Revealed

Evaluations on MechBench show a substantial gap between law discovery and mechanism discovery (Figure 5):

Codex (GPT-5.6-sol) : Core-set phenomenological law recovery = 35.00%; mechanism recovery = 13.75%. Even when the law is correctly recovered, mechanism recovery fails in 64.29% of cases. Providing the correct law directly only raises mechanism recovery to 45.00% (core) / 49.02% (full).

GPT-6-Astra : Law recovery jumps to 72.50% (core), but mechanism recovery reaches only 47.50%. Among tasks where GPT-6-Astra recovers the law correctly, 43.10% still fail full mechanism recovery. On high-difficulty problems with five mechanism mutations, law recovery is 50.00% but full mechanism recovery drops to 12.50%.

These results demonstrate that even the most powerful current models, while adept at discovering what laws govern data, remain far from explaining why those laws arise. MechBench quantifies this long-overlooked gap and sets a higher bar for future scientific agents.

Conclusion: From "Knowing What" to "Understanding Why"

SRHarness and MechBench correspond to two levels of AI for Science: reliable formula discovery and mechanistic understanding. Together they signal a shift from fitting observations to understanding mechanisms, and from single-shot formula generation to long-horizon scientific search.

Future competitive scientific agents will likely emerge from co-design of models, search strategies, and runtime frameworks — enabling reliable tool use, persistent hypothesis storage, effective evidence reuse, and objective mechanism verification. This may be the next station for AI for Science: not only knowing what the world is, but explaining why.

References

[1] Yu Z, Zhou S, Huang H, Ding J, Li Y. SRHarness: A Harness for Agentic Symbolic Regression. https://arxiv.org/abs/2609.35501 [2] Yu Z, Zhang J, Cheng J, Ding J, Li Y. MechBench: Can AI Scientific Agents Discover Mechanisms beyond Phenomenal Laws? https://arxiv.org/abs/2609.35515 [3] Ding J, Zheng Y, Xu F, et al. Understanding emergence in complex systems using abductive AI. Nature Reviews Physics . https://www.nature.com/articles/s42254-025-00895-5 [4] Yu Z, Ding J, Li Y. Discovering network dynamics with neural symbolic regression. Nature Computational Science .

https://www.nature.com/articles/s43588-025-00893-8