BestHub
Sign in
Tagged articles

LatentMoE

3 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jul 28, 2026 · Artificial Intelligence

The Most Valuable Innovations in Kimi K3’s 47‑Page Technical Report

Kimi K3 scales to 2.8 T parameters, 104 B activation parameters and a 1 M‑token context by introducing KDA‑based compressed sequence state, AttnRes for depth‑wise residual selection, Stable LatentMoE for efficient expert routing, partial‑rollout reinforcement learning and Firecracker micro‑VM sandboxing, achieving roughly 2.5× scaling efficiency and notable benchmark gains over competing models.

AttnResKDAKimi K3
0 likes · 17 min read
The Most Valuable Innovations in Kimi K3’s 47‑Page Technical Report
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 18, 2026 · Artificial Intelligence

NVIDIA Nemotron 3 Super: 7× Faster Than Qwen3.5 – Inside Hybrid Mamba‑Attention, LatentMoE, and MTP

NVIDIA’s Nemotron 3 Super, a 120.6 B‑parameter flagship model supporting 1 M‑token context, combines Hybrid Mamba‑Attention, LatentMoE, and Multi‑Token Prediction to achieve up to 7.5× higher inference throughput than Qwen3.5 while matching or surpassing its accuracy across a range of benchmarks.

Hybrid Mamba-AttentionLatentMoEMTP
0 likes · 11 min read
NVIDIA Nemotron 3 Super: 7× Faster Than Qwen3.5 – Inside Hybrid Mamba‑Attention, LatentMoE, and MTP
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Mar 12, 2026 · Artificial Intelligence

Nvidia’s Nemotron 3 Super Enters OpenClaw, Rivalling Opus 4.6

Nvidia unveiled the 120‑billion‑parameter Nemotron 3 Super, featuring a Mamba‑MoE hybrid architecture, LatentMoE routing, and Multi‑Token Prediction that together deliver up to 5× higher throughput and 3× faster inference, achieve 85.6% success on OpenClaw—matching Claude Opus 4.6 and GPT‑5.4—and set new records across Pinchbench, MMLU, SWE‑Bench, and other benchmarks, all while being fully open‑sourced with its training data and RL pipelines.

AI agentsLatentMoEMamba-MoE
0 likes · 14 min read
Nvidia’s Nemotron 3 Super Enters OpenClaw, Rivalling Opus 4.6