LeCun Team’s Triple Breakthrough: Sparse Representations, Gradient Planning, and Lightweight JEPA for World Models

Background and Challenges

Non‑generative world‑model approaches avoid pixel‑level generation cost but still suffer from three engineering and algorithmic bottlenecks: dense feature representations lacking biological sparsity, downstream planning that cannot fully exploit gradient information, and heavyweight architectures that are hard to reproduce on generic hardware.

Rectified LpJEPA: Sparse Representations

The paper “Rectified LpJEPA: Joint‑Embedding Predictive Architectures with Sparse and Maximum‑Entropy Representations” (arXiv:2602.01456) introduces a new regularizer called RDMReg (Rectified Distribution Matching Regularization) that forces the feature projection to match a Rectified Generalized Gaussian (RGG) distribution instead of the isotropic Gaussian used by VICReg. The RGG combines a Dirac component that explicitly controls the proportion of zero activations with a generalized‑Gaussian component that models the non‑zero values. By adjusting the shape and location parameters, researchers can precisely set the sparsity level while preserving a maximum‑entropy constraint, thereby achieving both high information content and a desired neuron silence rate.

Experiments on linear evaluation of ImageNet‑100 demonstrate that increasing sparsity does not degrade accuracy; the model maintains competitive classification performance while activating far fewer neurons, confirming that sparsity and performance can coexist.

Further analysis shows that matching the feature distribution with the sliced Wasserstein distance (SWD) forces the encoder to learn statistically independent, disentangled representations, which benefits downstream planning tasks.

GRASP: Gradient‑Based Planning

The second paper, “Parallel Stochastic Gradient‑Based Planning for World Models” (arXiv:2602.00475), argues that after obtaining efficient world‑model representations, the remaining difficulty lies in decision making. Traditional model‑predictive control relies on zero‑order optimizers such as CEM or MPPI, which sample many action sequences and scale poorly with high‑dimensional or long‑horizon tasks.

GRASP (Gradient Relaxed Stochastic Planner) replaces the serial sampling paradigm with a “configuration‑point” formulation, treating each timestep’s action as an independent optimization variable. The core objective minimizes a dynamics‑violation error, enabling parallel computation and shortening the gradient‑propagation path.

To avoid pathological gradient updates that would cheat by modifying the model’s internal state, GRASP introduces a gradient‑truncation mechanism: gradients are back‑propagated only through the action variables, not through the world‑model inputs. Additionally, Langevin dynamics noise is injected during state updates to prevent convergence to local minima and to encourage exploration.

Empirical results on long‑horizon tasks such as PointMaze navigation show that GRASP substantially outperforms CEM, achieving higher planning success rates.

EB‑JEPA: Lightweight Engineering

The third contribution, “A Lightweight Library for Energy‑Based Joint‑Embedding Predictive Architectures” (arXiv:2602.03604), focuses on reducing the entry barrier for JEPA‑based world models. Existing implementations of I‑JEPA or V‑JEPA require large compute budgets and are tightly coupled to specific infrastructure.

EB‑JEPA decouples the three core JEPA components—encoder, predictor, and loss (e.g., VICReg or SIGReg)—into interchangeable modules, allowing researchers to swap parts easily. The library is optimized for single‑GPU training; on a V100 (16 GB) the full training pipeline finishes in a few hours.

Despite its lightweight design, EB‑JEPA attains high‑quality representations: linear evaluation on CIFAR‑10 reaches 91 % accuracy, and the same codebase supports action‑conditioned video prediction (Action‑Conditioned Video‑JEPA). This makes it possible to validate new regularizers (such as Rectified LpJEPA) or planners (such as GRASP) without prohibitive resource costs.

Conclusion

Together, Rectified LpJEPA, GRASP, and EB‑JEPA close the loop on the JEPA world‑model stack: they advance sparse, high‑entropy representations; demonstrate the advantage of gradient‑based long‑horizon planning; and provide a modular, low‑cost software foundation for future research.