GPT-6 Astra Scores 98% on Physics Benchmark; MIT Reports 17.5x AI Gain

TheoPhysEval: A New Benchmark for Theoretical Physics

Researchers from top institutions including the International Centre for Theoretical Sciences (ICTS), Tata Institute of Fundamental Research (TIFR), University of Texas at Austin, University of Tokyo, Harvard University, University of Pennsylvania, Duke University, and Google DeepMind created TheoPhysEval , a benchmark of 50 unpublished theoretical physics problems drawn from experts' latest research notes. Contributors include Suvrat Raju, Sudip Chakraborty, and A. Ramesh Chandra. The problems span quantum field theory, holography, and other frontier areas, each requiring a day or more for a top specialist to solve.

Testing Setup and Models

The evaluation used two tracks: a Real Track allowing internet search and code execution, and an Isolated Track with no network access. Models tested were GPT-6 Astra, Claude Opus 5.5, Gemini 3.8 Flash, and DeepSeek-V4.1-Flash. Initial scores: Astra ~88%, Opus 5.5 92% on the Real Track.

AI Catches Human Errors

When experts reviewed Astra's supposedly incorrect answers, the model insisted its derivations were flawless and that the human answer keys contained mistakes. In multiple cases, experts re-examined their calculations and found subtle errors such as sign mistakes and misapplied approximation conditions. After correcting the answer keys, Astra's score rose to 98% (49 out of 50) .

Representative Solved Problems

Black-string Page time : In AdS4 with a two-brane black string background, Astra located the five-dimensional equations in literature, generalized them to three dimensions, wrote a Python script to handle singular differential equations, and produced the correct numerical Page time.

All-electron atomic solver : Astra built a self-consistent solver from scratch using Local Density Approximation (LDA), Dirac-Slater exchange, and VWN5 parametrization, solving the Kohn-Sham equations for boron and magnesium ground-state orbital energies matching NIST microhartree-level benchmarks.

Axion conversion rate : In natural units (ℏ = c = 1), Astra derived the exact photon production rate from axion-electron scattering in a dielectric medium without external fields.

Chern–Simons beta function : For 3D Chern-Simons matter theory, Astra identified propagators and vertices, performed minimal subtraction renormalization, and delivered a closed-form beta-function coefficient.

Other solved problems include boundary-layer perturbation scales in fluid dynamics, central charge of 1D D-class fermion systems, two-loop sub-divergence renormalization in 6D Euclidean scattering amplitudes, Seiberg duality in 14-node quiver gauge theories, and Euclidean wormhole action for IIB supergravity on T^6.

Cost and Isolated Track Performance

Average API cost per problem was under $3 . Even in the Isolated Track (no internet), Astra and Opus 5.5 both scored 92%, showing these models have internalized vast physics literature and deeply understand key formulas and logic. To prevent guessing, problems required converting analytic expressions into high-precision decimal strings with distractor floating-point values.

MIT Study: 17.5× Growth in Two Years

Neil Thompson's team at MIT tested o1, o3, and GPT-5 family models on problems extracted from papers published between December 2025 and May 2026 across probability theory, statistics, economic theory, high-energy physics, gravity and cosmology, and quantum physics. In mid-2024 the best AI solve rate was 2%; by mid-2026 it reached 35%, a 17.5× increase with no sign of slowing.

Implications for Scientific Practice

The paper argues that assigning verification calculations to junior researchers is no longer appropriate because AI does it better, more accurately, and more cheaply. Human "taste" — choosing problems, setting direction, and providing intuition — becomes paramount. The one problem Astra failed (Problem 1b: reconstructing an autonomous ODE from a noisy trajectory) illustrates the current limit: all top AIs failed because they lack physical intuition and cannot reverse-engineer when information is asymmetric.

Future Outlook

Research will shift to multi-turn human-AI interaction : humans supply inspiration and taste, AI handles derivation and massive computation. Long-standing challenges like grand unification and dark matter may fall within years.

References

arXiv:2610.16733 (https://arxiv.org/abs/2610.16733)

X post by Nirmalya Kajuri (https://x.com/Kaju_Nut/status/2110376532267196754)

Superpower Daily summary (https://superpowerdaily.com/posts/ai-success-on-selected-science-theory-problems-rose-from-2-to-35-mit-study-finds)