Skip to content
Tagged articles

grokking

8 articles · Page 1 of 1
PaperAgent
PaperAgent
Oct 25, 2026 · Artificial Intelligence

Google Discovers Two 20-Year-Old Bugs in Softmax-Cross-Entropy Implementation

Google researchers identify two numerical bugs in the standard softmax-cross-entropy implementation used for two decades: Softmax Collapse zeros out gradients beyond a precision cliff, and Softmax Cancellation creates a noise floor in distillation; fixes using stable log-space computations eliminate Slingshot loss spikes in grokking and improve convergence across precisions.

Adam optimizerSlingshotcross-entropy
0 likes · 8 min read
Google Discovers Two 20-Year-Old Bugs in Softmax-Cross-Entropy Implementation
Machine Heart
Machine Heart
Oct 10, 2026 · Artificial Intelligence

Tian Yuandong: Co-Authoring a Paper with GPT-5 Made Me Realize My Job Could Vanish in 5 Years

Meta FAIR veteran and Recursive Superintelligence co-founder Tian Yuandong recounts his journey from AlexNet debates to Go AI, Coconut, and grokking research, revealing how co-authoring a paper with GPT-5 boosted efficiency 6-10x and led him to predict AI will replace researchers within five years, while arguing humans must still initiate and verify ideas.

AI research automationCoconutGPT-5 collaboration
0 likes · 15 min read
Tian Yuandong: Co-Authoring a Paper with GPT-5 Made Me Realize My Job Could Vanish in 5 Years
DeepNoMind
DeepNoMind
Aug 30, 2026 · Interview Experience

System Design Interview Prep: 7 Resources & a 7-Week Roadmap to Offer

This article outlines a structured 7-week preparation plan for system design interviews, recommending seven key resources — from foundational primers to real-world case studies — and emphasizes deliberate practice over passive video consumption to master trade-off reasoning under pressure.

7-week planAlex XuByteByteGo
0 likes · 12 min read
System Design Interview Prep: 7 Resources & a 7-Week Roadmap to Offer
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jul 6, 2026 · Artificial Intelligence

ICML 2026 Opens – Tsinghua Wins Outstanding Paper, DeepMind Earns Test‑of‑Time Award, and Who Is Machine Learning For?

ICML 2026 in Seoul broke submission records, sparked controversy over LLM‑generated reviews, honored breakthrough papers on diffusion models, reinforcement learning and AI alignment, and culminated in a reflective question about the true purpose and beneficiaries of machine learning.

AI ethicsICML 2026Paper Awards
0 likes · 15 min read
ICML 2026 Opens – Tsinghua Wins Outstanding Paper, DeepMind Earns Test‑of‑Time Award, and Who Is Machine Learning For?
Machine Heart
Machine Heart
Jul 6, 2026 · Artificial Intelligence

ICML 2026 Awards Unveiled: Breakthroughs in Diffusion Models, AI Alignment, and Reinforcement Learning

ICML 2026 announced ten award‑winning papers, highlighting novel insights such as the flexibility trap in diffusion language models, high‑accuracy sampling for diffusion, risks of AI alignment tools, a random‑matrix view of diffusion consistency, grokking in ridge regression, and an asynchronous deep‑RL framework, each accompanied by concise abstracts and links.

AI AlignmentICML 2026Reinforcement Learning
0 likes · 16 min read
ICML 2026 Awards Unveiled: Breakthroughs in Diffusion Models, AI Alignment, and Reinforcement Learning
Open Source Tech Hub
Open Source Tech Hub
Mar 17, 2024 · Artificial Intelligence

What Is Grok? Inside Elon Musk’s New Open‑Source LLM and the ‘Grokking’ Phenomenon

Elon Musk announced the open‑source release of Grok, xAI’s new large‑language‑model chatbot, while recalling his lawsuit against OpenAI; the article explains Grok’s rapid development, links to the GitHub repository, summarizes the seminal “Grokking” research paper that describes a sudden generalization breakthrough in neural networks, and provides reference links.

AI researchGrokgrokking
0 likes · 3 min read
What Is Grok? Inside Elon Musk’s New Open‑Source LLM and the ‘Grokking’ Phenomenon
21CTO
21CTO
Aug 15, 2023 · Artificial Intelligence

Why Do Neural Networks Suddenly ‘Grok’ After Long Training? Insights from Google

Google’s recent research reveals that when small neural networks are trained for extended periods on tasks like modular addition, they can abruptly shift from memorizing training data to genuinely generalizing—a sudden “grokking” phenomenon driven by weight decay and the emergence of periodic weight structures.

AI researchGeneralizationMLP
0 likes · 9 min read
Why Do Neural Networks Suddenly ‘Grok’ After Long Training? Insights from Google
Architect
Architect
Apr 19, 2023 · Artificial Intelligence

Emergence in Large Language Models: Phenomena, Explanations, and Implications

This article reviews the emergence phenomena observed in large language models, explains how model scale, in‑context learning and chain‑of‑thought prompting contribute to sudden performance gains, discusses small‑model alternatives, and explores the relationship between emergence and the training‑time Grokking effect.

AI researchEmergenceModel Scaling
0 likes · 13 min read
Emergence in Large Language Models: Phenomena, Explanations, and Implications