Tagged articles

Muon optimizer

12 articles · Page 1 of 1

Machine Learning Algorithms & Natural Language Processing

Jul 30, 2026 · Artificial Intelligence

Who Built Kimi K3? Inside the Elite Team Driving a $70 B Valuation

The article profiles the 401‑person core team behind the open‑source 2.8‑trillion‑parameter Kimi K3 model, detailing their academic backgrounds, landmark papers, engineering breakthroughs such as Mooncake KV‑Cache, MoBA, Muon optimizer, and the performance gains that let K3 run at only 38% of Claude Fable 5’s cost while boosting request capacity by over 75%.

AI infrastructureKimi K3MoBA

0 likes · 37 min read

Who Built Kimi K3? Inside the Elite Team Driving a $70 B Valuation

Machine Learning Algorithms & Natural Language Processing

May 20, 2026 · Artificial Intelligence

Composer 2.5 Narrows the Gap to Claude Opus 4.7 with Ten‑Fold Cost Savings

Composer 2.5, the latest AI‑coding model from Cursor, claims near‑par performance with Claude 4.7 Opus and GPT‑5.5 while delivering up to ten‑times higher efficiency and a pricing model of $0.5 per M input tokens and $2.5 per M output tokens, backed by novel reinforcement‑learning tricks, massive synthetic data, and a custom Muon optimizer with dual‑grid HSDP architecture.

AI programmingComposer 2.5HSDP

0 likes · 13 min read

Composer 2.5 Narrows the Gap to Claude Opus 4.7 with Ten‑Fold Cost Savings

Machine Heart

May 18, 2026 · Artificial Intelligence

Composer 2.5 Delivers Opus‑level Performance at One‑Tenth the Cost

Composer 2.5, Cursor’s latest LLM, matches Claude Opus 4.7‑level capabilities while costing roughly one‑tenth as much, thanks to larger training scale, precise text‑feedback reinforcement learning, 25× more synthetic tasks, and a new Muon‑HSDP optimizer that boosts efficiency up to ten‑fold.

Composer 2.5LLMMuon optimizer

0 likes · 9 min read

Composer 2.5 Delivers Opus‑level Performance at One‑Tenth the Cost

Architect's Must-Have

Apr 28, 2026 · Artificial Intelligence

Why DeepSeek V4 Stands Apart: A Deep Dive into Its Architecture and Performance

DeepSeek V4 introduces a suite of architectural innovations—including mixed attention, manifold‑constrained hyper‑connections, the Muon optimizer, and FP4‑aware quantization—that together slash million‑token inference cost to a tenth of its predecessor while delivering benchmark results that rival top‑tier closed‑source models.

DeepSeek V4FP4 QuantizationMixture of Experts

0 likes · 44 min read

Why DeepSeek V4 Stands Apart: A Deep Dive into Its Architecture and Performance

DeepHub IMBA

Apr 27, 2026 · Artificial Intelligence

DeepSeek‑V4 Deep Dive: Engineering Million‑Token Context Efficiency

The article provides a thorough technical analysis of DeepSeek‑V4, detailing how mixed sparse attention (CSA + HCA), manifold‑constrained hyper‑connections, the Muon optimizer, FP4 quantization, and a suite of infrastructure tricks enable stable training and inference with up to one‑million token contexts while achieving state‑of‑the‑art benchmark results.

CSADeepSeek V4FP4 Quantization

0 likes · 22 min read

DeepSeek‑V4 Deep Dive: Engineering Million‑Token Context Efficiency

Baobao Algorithm Notes

Apr 27, 2026 · Artificial Intelligence

DeepDive into DeepSeek‑V4: Efficient Million‑Token Context, Hybrid Attention, and Muon Optimizer

The article provides an in‑depth technical analysis of DeepSeek‑V4, detailing its novel hybrid attention architecture (CSA and HCA), the manifold‑constrained hyper‑connection (mHC), massive KV‑cache reductions, FLOPs savings across token lengths, and the Muon optimizer with Newton‑Schulz orthogonalization, all backed by concrete benchmark tables and code snippets.

DeepSeekKV cache reductionMuon optimizer

0 likes · 61 min read

DeepDive into DeepSeek‑V4: Efficient Million‑Token Context, Hybrid Attention, and Muon Optimizer

TechVision Expert Circle

Apr 25, 2026 · Artificial Intelligence

DeepSeek V4 Arrives After 15 Months: Hybrid Attention Beats Expectations

DeepSeek's V4 preview, released 15 months after V3, introduces million‑token context, a hybrid attention architecture, and two model variants—Pro and Flash—offering significant compute savings, new training techniques, and performance gains that challenge top closed‑source LLMs.

DeepSeek V4FP4 QuantizationHybrid Attention

0 likes · 12 min read

DeepSeek V4 Arrives After 15 Months: Hybrid Attention Beats Expectations

ArcThink

Apr 25, 2026 · Artificial Intelligence

DeepSeek V4’s Silent Launch: 1.6 T Parameters, Triple Innovation, and Redefined Accessibility

DeepSeek V4 quietly debuted with a 1.6‑trillion‑parameter MoE model, introducing CSA+HCA compressed attention, mHC manifold‑constrained hyperconnections, and the Muon optimizer, achieving 1M‑token context at a quarter of V3’s cost, top Codeforces and LiveCodeBench scores, a 1/7 Opus price, MIT open‑source licensing, and dual‑stack Ascend NPU/NVIDIA GPU support.

DeepSeek V4Manifold-constrained HyperconnectionMixture of Experts

0 likes · 17 min read

DeepSeek V4’s Silent Launch: 1.6 T Parameters, Triple Innovation, and Redefined Accessibility

Machine Learning Algorithms & Natural Language Processing

Apr 25, 2026 · Artificial Intelligence

DeepSeek V4 Unveiled: 1M‑Token Context and New Architecture Challenge Closed‑Source LLMs

DeepSeek V4 introduces two flagship models—V4‑Pro with 1.6 T parameters and V4‑Flash with 284 B parameters—offering million‑token context, mixed attention (CSA + HCA), manifold‑constrained residuals, and the Muon optimizer, delivering open‑source performance that rivals top closed‑source LLMs while cutting inference cost dramatically.

1M contextDeepSeekMuon optimizer

0 likes · 10 min read

DeepSeek V4 Unveiled: 1M‑Token Context and New Architecture Challenge Closed‑Source LLMs

AI Explorer

Apr 24, 2026 · Artificial Intelligence

DeepSeek-V4 Raises the Bar: 1.6T‑Parameter Open‑Source Model Challenges Closed‑Source Giants

DeepSeek-V4 introduces two open‑source LLMs—V4‑Pro with 1.6 trillion total parameters and V4‑Flash with 284 billion—offering a 1 million‑token context window, hybrid attention, multi‑head compression, and a new Muon optimizer, all under an MIT license that rivals top closed‑source models.

DeepSeek V4Hybrid AttentionMulti-head Compression

0 likes · 6 min read

DeepSeek-V4 Raises the Bar: 1.6T‑Parameter Open‑Source Model Challenges Closed‑Source Giants

Machine Learning Algorithms & Natural Language Processing

Apr 4, 2026 · Artificial Intelligence

How Gram‑Newton‑Schulz Halves Muon Optimizer’s Compute Cost for Trillion‑Parameter Models

The article explains how the Muon optimizer’s expensive Newton‑Schulz orthogonalization is accelerated by the Gram‑Newton‑Schulz algorithm, which reduces end‑to‑end orthogonalization time by 40‑50%, achieves up to 2× speed‑up in large‑scale LLM training, and resolves numerical stability issues through a restart strategy and custom GPU kernels.

GPU kernelsGram Newton-SchulzMuon optimizer

0 likes · 9 min read

How Gram‑Newton‑Schulz Halves Muon Optimizer’s Compute Cost for Trillion‑Parameter Models

Baobao Algorithm Notes

Jul 17, 2025 · Artificial Intelligence

How QK-Clip Tames MaxLogit Explosions in Trillion‑Parameter LLMs

The article introduces QK-Clip, a lightweight per‑head weight‑clipping technique that uses the MaxLogit signal to prevent uncontrolled logit growth in massive LLMs, explains its design, compares it with prior methods, and shows that it stabilizes training without harming model performance.

Attention stabilityLLM trainingMaxLogit

0 likes · 15 min read

How QK-Clip Tames MaxLogit Explosions in Trillion‑Parameter LLMs