Has Deep Learning Discovered Its Own “Newton’s Law”?

Deep learning currently lacks a solid scientific theory; breakthroughs come from engineering intuition while pioneers like LeCun and Hinton describe the field as a barren theoretical landscape.

A team of 14 researchers from UC Berkeley, Harvard, Stanford and other institutions recently published a paper (arXiv:2604.21691) called There Will Be a Scientific Theory of Deep Learning , introducing the term Learning Mechanics for a nascent theoretical framework.

The authors liken this effort to the historical unifications in physics—classical mechanics, statistical mechanics, quantum mechanics—suggesting that Learning Mechanics could become a first‑principles description of neural network learning.

Historically, major advances such as AlexNet, ResNet, and the Transformer arose from empirical discovery rather than theory, leaving practitioners to rely on experience and luck when tuning models.

The paper identifies five research strands that together point toward a unified theory:

Solvable idealized settings : In simplified models like deep linear networks, SGD provably finds the global optimum and its trajectory can be analytically described, mirroring the solvable harmonic‑oscillator and hydrogen‑atom models in physics.

Tractable limits : When network width, depth, batch size, or learning rate approach extreme values, behavior becomes predictable—e.g., the lazy (kernel‑like) versus rich (feature‑learning) regimes in the infinite‑width limit, analogous to thermodynamic limits.

Empirical laws : Universal scaling laws (loss ∝ compute⁻ᵅ) hold across architectures and tasks, and the Edge of Stability (gradient’s top eigenvalue stabilizing near 2/η) resembles Snell’s law in optics.

Hyperparameter theory : Concepts such as μP (Maximal Update Parameterization), central flow, and hyperparameter decoupling provide a dimensional‑analysis‑style framework that enables zero‑sample transfer of hyperparameters across model scales.

Universal behavior : Representation convergence shows that vastly different networks (e.g., ResNet and Vision Transformer) learn remarkably similar internal features, echoing critical universality in statistical mechanics.

These strands converge toward a single unified framework that could serve as the “periodic table” of deep learning.

The authors also list ten open problems, including analytic theory for nonlinear dynamics, origins of scaling laws, a complete phase diagram of lazy versus rich regimes, a standard model for hyperparameters, rigorous proofs of representation convergence, theoretical bounds on generalization error, principled architecture design, mechanisms behind emergent language and reasoning, connections between physical symmetries and neural inductive biases, and a formal axiomatic system for Learning Mechanics.

In summary, the paper argues that deep learning is at a pivotal moment similar to chemistry before Lavoisier—rich in empirical recipes but lacking a unifying theory—and that assembling the identified pieces may finally provide the scientific foundations for the field.