Skip to content
Tagged articles

AdaGrad

3 articles · Page 1 of 1
Machine Heart
Machine Heart
Oct 18, 2026 · Artificial Intelligence

Elad Hazan's 20-Year OCO Retrospective: Regret Minimization, AdaGrad, and AI Frameworks

Princeton professor Elad Hazan recounts 20 years of Online Convex Optimization research, from inventing Online Newton Step and co-creating AdaGrad to developing spectral filtering for dynamical systems and Flash STU architectures, arguing that as AI excels at theorem-proving, human value lies in choosing problems and inventing frameworks.

AdaGradDeep LearningOnline Convex Optimization
0 likes · 18 min read
Elad Hazan's 20-Year OCO Retrospective: Regret Minimization, AdaGrad, and AI Frameworks
Hulu Beijing
Hulu Beijing
Jan 4, 2018 · Artificial Intelligence

Why SGD Fails and How Momentum, AdaGrad, and Adam Fix It

This article explains why vanilla Stochastic Gradient Descent often struggles in deep learning, describes the challenges of valleys and saddle points, and introduces three major SGD variants—Momentum, AdaGrad, and Adam—detailing their motivations, update rules, and advantages.

AdaGradAdamOptimization
0 likes · 13 min read
Why SGD Fails and How Momentum, AdaGrad, and Adam Fix It