LeCun at CoRL 2026: Abandon Four Research Paths — World Models, Not LLMs, Are Key to Human-Level AI
Yann LeCun argues intelligence is rapid adaptation to the unknown, not knowledge accumulation. He presents JEPA and hierarchical H-JEPA as world models that predict action consequences in abstract representation spaces, and urges researchers to drop generative modeling, probabilistic models, contrastive learning, and reinforcement learning in favor of energy-based models, regularization, and model-predictive control.
At CoRL 2026, Yann LeCun delivered a talk titled "World Models: Enabling the Next AI Revolution," arguing that current AI systems lack the foundational world understanding that enables humans to learn new tasks rapidly. A 17‑year‑old learns to drive in hours; autonomous driving systems have consumed millions of hours of data yet have not achieved Level 5 autonomy. LeCun defines intelligence as the speed of finding solutions to novel problems, quoting Piaget: "Intelligence is not what you know, but what you do when you don't know."
World Models Should Predict Consequences, Not Generate Video
LeCun distinguishes world models from video generators or full physics simulators. A world model takes a current abstract state s(t) and a hypothetical action a(t) and predicts the next abstract state s(t+1). Its purpose is to support planning by searching over action sequences that minimize a task objective while satisfying safety constraints ("Guardrail Objectives"). This mirrors Model Predictive Control (MPC) but uses learned, differentiable world models.
Hierarchical Abstraction for Long‑Horizon Planning
Planning at the lowest control frequency (e.g., 20–50 Hz joint torques) is intractable for long tasks. LeCun advocates hierarchical world models (H‑JEPA) where higher levels predict over longer time spans with coarser abstractions, providing sub‑goals to lower levels. In the AntMaze benchmark, H‑JEPA uses three levels predicting 5, 10, and 20 environment steps respectively. Higher‑level representations yield smoother value functions, making planning easier.
JEPA: Joint Embedding Predictive Architecture
Generative models predict pixels, wasting capacity on unpredictable details (e.g., leaves fluttering, unseen faces). JEPA predicts in an abstract representation space. An encoder maps inputs to representations; a predictor forecasts future representations. The core challenge is avoiding representation collapse — where the encoder ignores input and outputs a constant. LeCun reviews several regularization approaches:
VICReg / Barlow Twins: variance and decorrelation constraints on representation dimensions.
Clustering methods: linear projection + Softmax toward cluster centers with distribution constraints.
SIGReg (Sketched Isotropic Gaussian Regularization): projects batch representations onto random directions and matches marginal distributions to Gaussians via characteristic functions or CDF comparisons. SIGReg works on simple world‑model tasks but assumes latent variables are Gaussian, which may not hold generally.
LeCun emphasizes the trade‑off: maximize information content while minimizing prediction error, so the model keeps predictable structure and discards chaotic noise.
Self‑Supervised Video Learning Yields 3D Structure Without Labels
V‑JEPA 2.1, trained purely by predicting masked/future video patches in representation space, learns representations that contain depth, object occlusion ordering, and other 3D spatial information. A linear probe on frozen representations predicts depth accurately despite never seeing depth labels. LeCun explains: camera motion induces parallax and occlusion changes; predicting these changes forces the model to internalize 3D geometry.
Four Research Direction Shifts
Abandon generative modeling → adopt joint‑embedding architectures (JEPA). Learn abstract representations that support prediction, not pixel reconstruction.
Abandon probabilistic models → adopt energy‑based models. Energy‑based frameworks better capture the structure needed for planning and reasoning.
Abandon contrastive learning → adopt regularization methods. Contrastive methods require large batches and negative samples; regularization (VICReg, SIGReg, clustering) avoids collapse more efficiently.
Abandon reinforcement learning as the primary paradigm → adopt model‑predictive control (MPC). RL can still fine‑tune world models or critics when predictions mismatch reality, but MPC with learned world models should be the core for physical interaction.
Final Position on LLMs and AGI
LeCun dismisses "Artificial General Intelligence" as a misleading term — even human intelligence is highly specialized. He asserts that achieving human‑level AI cannot rely on scaling LLMs alone. Instead, the field must build world models (JEPA, H‑JEPA) that enable machines to understand physical reality, predict action outcomes, and plan hierarchically.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
