Skip to content
Tagged articles

MoE

112 articles · Page 1 of 2
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 16, 2026 · Artificial Intelligence

CPU-Only 284B LLM at 9 Tokens/sec: Why Memory Capacity Now Trumps GPU

The llama-modchip project demonstrates running a 284B parameter MoE model on a 64GB RAM CPU-only machine via aggressive quantization to 56.6GB and kernel optimizations, but achieves only 9 tokens/sec decoding speed, revealing that memory capacity—not GPU—is becoming the key bottleneck for local large language model inference, with practical implications for Mac memory sizing.

CPU inferenceMac hardwareMoE
0 likes · 11 min read
CPU-Only 284B LLM at 9 Tokens/sec: Why Memory Capacity Now Trumps GPU
IT Services Circle
IT Services Circle
Nov 16, 2026 · Artificial Intelligence

Anthropic's FlashProve: Zero-Knowledge Proofs for LLM Model Verification

Anthropic's open-source FlashProve project uses zkSNARK zero-knowledge proofs to let users verify that LLM responses come from the claimed model, detecting proxy services that secretly swap expensive models for cheaper alternatives, though practical deployment faces cost, context-length, and trust-anchor challenges.

AI proxyAnthropicFlashProve
0 likes · 17 min read
Anthropic's FlashProve: Zero-Knowledge Proofs for LLM Model Verification
Architects' Tech Alliance
Architects' Tech Alliance
Nov 15, 2026 · Artificial Intelligence

First 10K-Card AI Cluster Network Report: Domestic IB Breakthrough at 100K Scale

The article analyzes China's first ten-thousand-card intelligent computing cluster network technology report, detailing how Sugon's scaleFabric domestic InfiniBand native RDMA system overcomes scaling bottlenecks through hardware-native lossless RDMA, hierarchical supernode/supercluster architecture, large subnet expansion to 114,623 cards, and high-reliability multi-tenant operations, validated on the 100K-card 曙光8000 super-intelligent fusion cluster.

AI clusterInfiniBandMoE
0 likes · 19 min read
First 10K-Card AI Cluster Network Report: Domestic IB Breakthrough at 100K Scale
DataFunSummit
DataFunSummit
Nov 15, 2026 · Artificial Intelligence

284B DeepSeek on 64GB CPU: Quantization Works, 96GB Xeon Fails on AVX-512

The llama-modchip project runs a 284B-parameter DeepSeek-V4-Flash model on a 64GB RAM CPU-only machine via aggressive quantization to 1.71 bits/weight (56.56 GiB), achieving 9–10 tokens/sec, but a 96GB Xeon server fails due to custom GGUF tensor types and missing AVX-512, highlighting hardware-software compatibility gaps.

AVX-512CPU inferenceDeepSeek-V4-Flash
0 likes · 13 min read
284B DeepSeek on 64GB CPU: Quantization Works, 96GB Xeon Fails on AVX-512
AI Frontier Lectures
AI Frontier Lectures
Nov 14, 2026 · Backend Development

llama.cpp 1.0 Drops 35 Deprecated APIs, Adopts Semantic Versioning

llama.cpp 1.0.0 removes 35 deprecated functions from its public C API, adopts semantic versioning where only breaking API changes trigger major versions, adds FP8 KV cache support, MoE CPU offloading, new model integrations including Kimi-K3.1-Pro and drama-3, and provides pre-built binaries for 150+ architecture variants across major platforms.

C APIFP8KV Cache
0 likes · 5 min read
llama.cpp 1.0 Drops 35 Deprecated APIs, Adopts Semantic Versioning
SmartApe AI
SmartApe AI
Nov 13, 2026 · Artificial Intelligence

No Second Run: RSI for LLM Pretraining Yields 30B MoE Model in 3 Months

Shanghai AI Lab and Xinxu AI introduce No Second Run, a recursive self-improvement method for LLM pretraining using debugging, headroom evaluation, and experience-to-metric loops. In three months they trained the 801-30B-A3B MoE model (30B total, 3B active params) on 11.8T tokens, scoring 73.13 average on 13 base benchmarks (second only to Qwen3.5-35B-A3B) and achieving top Agent benchmark results after 6.33B-token SFT without RL.

801-30B-A3BAgent BenchmarksLLM Pretraining
0 likes · 15 min read
No Second Run: RSI for LLM Pretraining Yields 30B MoE Model in 3 Months
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 10, 2026 · Artificial Intelligence

Mac Studio + RTX 5090 eGPU Runs Qwen3.8-Flash-Next at 210 tok/s, 2.5x M4 Max

An open-source project called Apple NVIDIA Sidecar enables a Mac Studio with an RTX 5090 eGPU to run the 125B MoE model Qwen3.8-Flash-Next at 210 tokens per second, outperforming a standalone M4 Max and a single 5090 PC, but requires compiling a custom driver and only supports one session at a time.

AI inferenceApple NVIDIA SidecarMac Studio
0 likes · 7 min read
Mac Studio + RTX 5090 eGPU Runs Qwen3.8-Flash-Next at 210 tok/s, 2.5x M4 Max
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 6, 2026 · Artificial Intelligence

M6 Launch Day: 9.1 vs 10.6 tok/s — Should You Upgrade 16GB to 32GB for Local LLMs?

Real-world tests show M6 MacBook Pro hits 9.1 tok/s on Qwen3.8-27B 4-bit, reaching 86% of the 10.6 tok/s bandwidth ceiling; the article explains why the dual neural engine doesn't accelerate token generation, debunks the 8-model limit as a software filter, and provides a three-tier buying guide based on memory bandwidth and model size.

Apple SiliconBuying GuideLLM inference
0 likes · 13 min read
M6 Launch Day: 9.1 vs 10.6 tok/s — Should You Upgrade 16GB to 32GB for Local LLMs?
SmartApe AI
SmartApe AI
Nov 5, 2026 · Artificial Intelligence

Baidu's ERNIE-5.5-Flash-Base: 280B MoE Hits Same-Size SOTA with 14B Active Params

Baidu open-sources ERNIE-5.5-Flash-Base, a 280B-parameter sparse MoE model activating only 14B parameters per token, trained on 33.5T tokens with 128K context, achieving same-size SOTA against DeepSeek-V4-Flash-Base and others through hybrid sparse attention, multi-router latent MoE, and ECHO future-token prediction.

BaiduECHOERNIE-5.5-Flash-Base
0 likes · 19 min read
Baidu's ERNIE-5.5-Flash-Base: 280B MoE Hits Same-Size SOTA with 14B Active Params
PaperAgent
PaperAgent
Nov 3, 2026 · Artificial Intelligence

FlashProve: Verifying LLM Outputs Match the Specified Model Without Revealing Weights

Anthropic's FlashProve introduces a cryptographic protocol that lets clients verify LLM outputs were generated by a specific committed model, using weight commitments, zero-knowledge proofs, and interactive oracle proofs, enabling verification on consumer hardware without re-running the model.

AnthropicFlashProveLLM verification
0 likes · 8 min read
FlashProve: Verifying LLM Outputs Match the Specified Model Without Revealing Weights
Lao Guo's Learning Space
Lao Guo's Learning Space
Nov 3, 2026 · Artificial Intelligence

Qwen4 27B vs Hy4 770B: Local Deployment Hardware Reality

This analysis compares Qwen4's anticipated 27B model with Hunyuan Hy4's 770B MoE, showing Hy4 demands an 8-GPU cluster (385GB+ VRAM) impossible for consumer hardware, while Qwen4 27B runs on a single 24-32GB GPU, making Qwen the only viable local choice.

Hunyuan Hy4LLMMoE
0 likes · 8 min read
Qwen4 27B vs Hy4 770B: Local Deployment Hardware Reality
Machine Heart
Machine Heart
Oct 22, 2026 · Artificial Intelligence

Flint-40B-A2B: AI-Built MoE Coding Model Hits 125 tok/s on MacBook Pro

Recursive_SI's first model Flint-40B-A2B, a 40B MoE model activating 2.5B parameters per token, was built in under four months using the automated Eureka Machine system. It achieves state-of-the-art local coding performance on benchmarks like SWE-Bench while running at 125+ tokens/sec on MacBook Pro M4 Pro and 150+ tokens/sec on NVIDIA RTX Spark, with open weights on Hugging Face under Apache 2.0.

Apple SiliconEureka MachineFlint-40B-A2B
0 likes · 10 min read
Flint-40B-A2B: AI-Built MoE Coding Model Hits 125 tok/s on MacBook Pro
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 21, 2026 · Artificial Intelligence

DeepSeek Open-Sources DeepInfer Inference Engine & deepseek-recipe API Adapter

DeepSeek has open-sourced DeepInfer, a high-performance inference framework supporting distributed inference, speculative decoding, and SWA Bounded Replay for MoE models, alongside deepseek-recipe, a production-grade token encoding/decoding component that standardizes API request handling for consistent deployment.

API adapterDeepInferDeepSeek
0 likes · 9 min read
DeepSeek Open-Sources DeepInfer Inference Engine & deepseek-recipe API Adapter
Lao Guo's Learning Space
Lao Guo's Learning Space
Oct 21, 2026 · Artificial Intelligence

Splash 1.4.0: 120 tok/s on 64GB M5 Pro, 3x Faster Than Ollama

Inco AI's open-source inference engine Splash 1.4.0 achieves 120 tokens/second decode and 1.3K tokens/second prefill on a 64GB M5 Pro running Qwen3.8-Flash-Next, outperforming Ollama by 3x and Strata by 30%, with MoE activation efficiency enabling large models on modest memory.

Apple SiliconInco AIM5 Pro
0 likes · 7 min read
Splash 1.4.0: 120 tok/s on 64GB M5 Pro, 3x Faster Than Ollama
Alibaba Cloud Native
Alibaba Cloud Native
Oct 20, 2026 · Artificial Intelligence

Inference Frameworks Demystified: Core Concepts, Frameworks & Cluster Deployment for LLMs

This comprehensive guide explains inference frameworks for large language models, covering core concepts like Prefill/Decode and KV Cache, comparing major open-source frameworks (vLLM, SGLang, TensorRT-LLM), detailing request processing flow from scheduling to sampling, and identifying who benefits from understanding inference infrastructure.

KV CacheLLM servingMoE
0 likes · 46 min read
Inference Frameworks Demystified: Core Concepts, Frameworks & Cluster Deployment for LLMs
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Oct 19, 2026 · Artificial Intelligence

PRIME: Joint Pruning & Merging for MoE Expert Compression with 2.23× Throughput Gain

This article presents PRIME, a unified integer linear programming approach that jointly decides per-expert retention, merging, or pruning based on activation frequency and output similarity, reducing routed-expert count by 25% with only ~1% accuracy drop on code/math benchmarks, and demonstrates up to 2.23× throughput improvement in high-concurrency long-output serving via working-set reduction and KV-capacity expansion.

MoEexpert compressioninteger linear programming
0 likes · 21 min read
PRIME: Joint Pruning & Merging for MoE Expert Compression with 2.23× Throughput Gain
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Oct 15, 2026 · Artificial Intelligence

Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations

Alibaba Cloud's Zhenwu M890 supernode (64 chips, 800GB/s interconnect, 9TB unified memory) adapted StepFun's Step 5 Preview (600B sparse MoE, 27B active, 1M context, multimodal) with operator fusion, expert parallel optimization, and prefill-decode separation, achieving lower TTFT, higher decode throughput, and stable 1M context support.

Inference OptimizationLong ContextMoE
0 likes · 6 min read
Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations
Architecture Digest
Architecture Digest
Oct 9, 2026 · Artificial Intelligence

How Strata Runs a 125B Model on a 12GB Gaming GPU

Strata is an open-source inference engine that enables running the 125B-parameter Qwen3.8-Flash-Next MoE model on consumer gaming PCs with 12GB VRAM by tiering experts across GPU, RAM, and SSD, using speculative decoding for 1.6-1.8x speedup, and providing OpenAI/Anthropic-compatible APIs for local coding agents.

GPU memory offloadingMoEQwen3.8-Flash-Next
0 likes · 14 min read
How Strata Runs a 125B Model on a 12GB Gaming GPU
SmartApe AI
SmartApe AI
Oct 8, 2026 · Artificial Intelligence

ID Balancing: PID Control Stabilizes Extremely Sparse MoE with 768 Experts

Qwen researchers propose ID Balancing, a PID-based load control method for extremely sparse MoE models, achieving stable expert load with 768 experts and Top-3 routing, reducing MaxVio by over 50% and MinVio by 12% while maintaining competitive language modeling and downstream task performance.

DeepSeekMixture of ExpertsMoE
0 likes · 11 min read
ID Balancing: PID Control Stabilizes Extremely Sparse MoE with 768 Experts
Data Bricklaying Diary
Data Bricklaying Diary
Oct 2, 2026 · R&D Management

Mission Engineering: Who Decides If the Coffee Is Good? Defining Success Metrics Beyond Timing

This article uses a morning coffee scenario to illustrate mission engineering's three-tier metric framework (MOS, MOE, MOP), showing how to define verifiable success criteria, separate stakeholder acceptance from process completion, and create traceable metric definitions that link overall outcomes to specific performance measurements.

MOPMOSMission Engineering
0 likes · 22 min read
Mission Engineering: Who Decides If the Coffee Is Good? Defining Success Metrics Beyond Timing
Data STUDIO
Data STUDIO
Sep 30, 2026 · Artificial Intelligence

DeepSeek Releases 6 Ascend AI Kernels: TileLang, DeepGEMM, FlashMLA & More

DeepSeek open-sources six low-level libraries for Huawei Ascend 950, covering operator DSL, matrix multiplication, MoE communication, sparse attention, and TopK selection, with reported near-peak hardware performance but requiring version-specific integration.

AI kernelsDeepEPDeepGEMM
0 likes · 14 min read
DeepSeek Releases 6 Ascend AI Kernels: TileLang, DeepGEMM, FlashMLA & More
Ops Development & AI Practice
Ops Development & AI Practice
Sep 26, 2026 · Artificial Intelligence

Local LLM Hardware Truths: MoE, CPU/GPU/NPU, and 32GB Mac Mini Tuning

This article explains why CPUs struggle with MoE model prefill but excel at decode, compares dense, SSM, MLA, and BitNet architectures, evaluates CPU, NVIDIA GPU, Apple UMA, and NPU hardware for local LLM inference, and provides a practical tuning guide for running 32B models on a 32GB Mac mini with MLX and Ollama.

Apple SiliconKV CacheLLM inference
0 likes · 21 min read
Local LLM Hardware Truths: MoE, CPU/GPU/NPU, and 32GB Mac Mini Tuning
Architecture Digest
Architecture Digest
Sep 21, 2026 · Artificial Intelligence

3 AI Open-Source Projects: DeepSeek Agent Runtime, Verified Diagrams, 744B MoE on 25GB RAM

This article reviews three cutting-edge AI open-source projects: DeepSeek's plugin-based Agent runtime framework (deepseek-harness), archify for generating verifiable architecture diagrams directly in coding agents, and colibri, a pure C inference engine enabling 744B MoE models to run on 25GB RAM via disk streaming.

DeepSeekMoEPlugin Architecture
0 likes · 8 min read
3 AI Open-Source Projects: DeepSeek Agent Runtime, Verified Diagrams, 744B MoE on 25GB RAM
AI Architecture Path
AI Architecture Path
Sep 17, 2026 · Artificial Intelligence

Run 744B MoE Model on 25GB RAM: Colibri's Tiered Storage Breakthrough

Colibri, a pure C inference engine with zero dependencies, enables running the 744B parameter GLM-5.2 MoE model on consumer hardware with just 25GB RAM and NVMe SSD by leveraging MoE sparsity and a three-tier storage scheduling system across VRAM, RAM, and disk, achieving 0.05–6.8 tok/s depending on hardware.

ColibriGLM-5.2MoE
0 likes · 14 min read
Run 744B MoE Model on 25GB RAM: Colibri's Tiered Storage Breakthrough
Architecture Digest
Architecture Digest
Sep 13, 2026 · Artificial Intelligence

Running 2.78T-Parameter Kimi K3 on 8GB RAM: Pure C Inference Engine Explained

A pure C inference engine runs the 2.78 trillion parameter Kimi K3 model on just 8GB RAM by streaming weights from disk, leveraging MoE sparsity (only 3.7% active per token) and computing directly on compressed formats, achieving correct output at 32.69 seconds per token on consumer hardware.

C languageKimi K3LLM compression
0 likes · 11 min read
Running 2.78T-Parameter Kimi K3 on 8GB RAM: Pure C Inference Engine Explained
Architects' Tech Alliance
Architects' Tech Alliance
Sep 11, 2026 · Artificial Intelligence

scaleFabric: IBGDA-Powered GPU-Autonomous Communication Validated at 100K-GPU Scale

Sugon's scaleFabric achieves China's first large-scale IBGDA deployment, letting GPUs directly drive InfiniBand NICs without CPU involvement; validated in 100K-GPU clusters, it eliminates the CPU penalty for small-packet MoE All-to-All communication, cuts latency by two orders of magnitude, and integrates NVMe-over-RDMA and XPU Direct Storage for end-to-end Token acceleration.

All-to-AllGPUDirect RDMAIBGDA
0 likes · 11 min read
scaleFabric: IBGDA-Powered GPU-Autonomous Communication Validated at 100K-GPU Scale
Old Zhang's AI Learning
Old Zhang's AI Learning
Sep 10, 2026 · Artificial Intelligence

DeepSeek-V4.1-Flash: 8B Activated Model Beats 1.6T V4-Pro, Local Deployment Tested

DeepSeek-V4.1-Flash open-sourced with CED architecture and 8B/16B activated parameters outperforms its 1.6T predecessor V4-Pro on coding benchmarks, approaches GPT-6 Astra on DeepSWE, but requires 510GB FP8 weights needing 8×H200 for full-context local deployment; author tests across six harnesses finding Claude Code/Codex integration near peak performance.

AI model evaluationCED architectureDeepSWE
0 likes · 8 min read
DeepSeek-V4.1-Flash: 8B Activated Model Beats 1.6T V4-Pro, Local Deployment Tested
Huawei Cloud Developer Alliance
Huawei Cloud Developer Alliance
Sep 8, 2026 · Artificial Intelligence

AReaL v1.0.5 LoRA RL: Low-Rank Adaptation for Accessible Large Model RL on Ascend

This article details AReaL-Ascend v1.0.5's LoRA RL capabilities, explaining how low-rank adaptation reduces memory overhead for large model reinforcement learning, describing two Megatron LoRA weight update modes (adapter sync vs. merge), and covering cross-node LoRA RL, MoE support, XCCL communication, and Qwen3.6-27B examples for practical deployment.

AReaLAscend NPULoRA
0 likes · 6 min read
AReaL v1.0.5 LoRA RL: Low-Rank Adaptation for Accessible Large Model RL on Ascend
Cambridge Mofang Notes
Cambridge Mofang Notes
Sep 1, 2026 · Artificial Intelligence

From Dense to MoE: Decoding Total vs. Activated Parameters

This article explains the distinction between total and activated parameters in Mixture-of-Experts (MoE) models, contrasting dense and sparse architectures, detailing expert routing mechanisms, and analyzing memory and compute implications across model loading, prefill, and decode stages.

Mixture of ExpertsMoESparse Models
0 likes · 15 min read
From Dense to MoE: Decoding Total vs. Activated Parameters
PaperAgent
PaperAgent
Aug 22, 2026 · Artificial Intelligence

DeepSeek’s Hidden Multimodal Model: Technical Deep‑Dive and Unexpected Bugs

The article reviews DeepSeek‑V4‑Flash‑Vision‑Exp, exposing a misidentification bug, detailing its visual‑primitive approach, impressive spatial‑reasoning benchmarks, and a highly compressed KV‑cache architecture that balances performance with efficiency.

DeepSeekKV Cache compressionMoE
0 likes · 4 min read
DeepSeek’s Hidden Multimodal Model: Technical Deep‑Dive and Unexpected Bugs
AntTech
AntTech
Aug 20, 2026 · Artificial Intelligence

Ling-3.0-flash: Open-Source LLM Designed for Real-World Deployment

Ling-3.0-flash is a newly open‑sourced 124B‑parameter MoE model that offers multiple quantized versions, API, single‑machine private deployment, and high‑performance GPU inference exceeding 1100 tokens/s, with detailed benchmarks, optimization techniques, and real‑world use‑case analyses for agents, coding, and sensitive data processing.

Agent applicationsLLMLing-3.0-flash
0 likes · 15 min read
Ling-3.0-flash: Open-Source LLM Designed for Real-World Deployment
AI Architecture Path
AI Architecture Path
Aug 7, 2026 · Artificial Intelligence

Run 2.8‑Trillion‑Parameter Models on a 4 GB GPU with AirLLM’s Lossless Inference

AirLLM lets ordinary 4‑GB consumer GPUs run massive LLMs such as 70B Llama and the 2.8‑trillion‑parameter Kimi K3 MoE without quantisation, by streaming model layers from disk, offering a detailed comparison with traditional low‑VRAM tricks, step‑by‑step installation, pitfalls, scenario‑based guidance and an assessment of strengths and risks.

AirLLMLLM inferenceMoE
0 likes · 16 min read
Run 2.8‑Trillion‑Parameter Models on a 4 GB GPU with AirLLM’s Lossless Inference
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 5, 2026 · Artificial Intelligence

Sand.ai Releases First 100B‑Parameter MoE Video Model – 10‑Sec 1080p for $0.05

Sand.ai open‑sourced MAGI‑2‑preview, a 114‑billion‑parameter video generation model that activates only 6 billion parameters per inference, achieving 10‑second 1080p output for just five‑tenths of a yuan and ranking sixth on the AA video benchmark, while detailing the MoE‑based scaling challenges and the custom infrastructure that makes it feasible.

MAGI-2-previewMoElarge-scale models
0 likes · 11 min read
Sand.ai Releases First 100B‑Parameter MoE Video Model – 10‑Sec 1080p for $0.05
21CTO
21CTO
Aug 4, 2026 · Artificial Intelligence

How DeepSeek’s Cutting‑Edge Tech and Founder Control Power Drive Its IPO Plans

DeepSeek has begun IPO preparation targeting a 2027 listing, possibly as early as year‑end, backed by a $1.5 billion financing round that lifts its valuation to $71 billion, while its founder retains roughly 78% of equity and the company showcases a self‑developed, cost‑efficient AI stack.

AIDeepSeekDualPipe
0 likes · 7 min read
How DeepSeek’s Cutting‑Edge Tech and Founder Control Power Drive Its IPO Plans
AI Engineering
AI Engineering
Aug 2, 2026 · Artificial Intelligence

Kimi K3 Technical Report Reveals Answers to Key Architecture Questions

The 47‑page Kimi K3 technical report, released on July 27, details the 2.8‑trillion‑parameter model’s novel LatentMoE, SiTU‑GLU, Quantile Balancing, Attention Residuals, and full‑stack NoPE design, explains how these solve activation‑explosion and load‑imbalance problems, and provides open‑source code for inference and agentic RL.

Attention ResidualsKimi K3LatentMoE
0 likes · 12 min read
Kimi K3 Technical Report Reveals Answers to Key Architecture Questions
Machine Heart
Machine Heart
Jul 29, 2026 · Artificial Intelligence

How Ling‑3.0‑flash Proves “Less Is More” with 124B Parameters but Only 5.1B Activated

Ling‑3.0‑flash demonstrates that a 124‑billion‑parameter model can achieve flagship‑level performance while activating only 5.1 billion parameters, thanks to native mixed‑linear attention, KDA, and extreme MoE sparsity, making it a fast, cost‑effective execution engine for Agent‑centric workflows.

Agent ExecutionLing-3.0-flashMoE
0 likes · 17 min read
How Ling‑3.0‑flash Proves “Less Is More” with 124B Parameters but Only 5.1B Activated
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Jul 28, 2026 · Artificial Intelligence

Day0 Adaptation of Kimi K3 on Alibaba Cloud Lingjun Zhenwu M890 Supernode

On July 27, Alibaba Cloud announced that its Lingjun Zhenwu M890 supernode instance has been successfully adapted to run the 2.8‑trillion‑parameter Kimi K3 model, achieving a 35 % reduction in first‑token latency, a 1.8× increase in decode throughput, and support for up to 1 M token context through joint chip, software‑stack and framework optimizations.

Inference OptimizationKDAKimi K3
0 likes · 6 min read
Day0 Adaptation of Kimi K3 on Alibaba Cloud Lingjun Zhenwu M890 Supernode
AntTech
AntTech
Jul 27, 2026 · Artificial Intelligence

LLaDA2.2 Released: Levenshtein Editing Enables Diffusion Language Models to Correct On-the-Fly

LLaDA2.2 introduces a Levenshtein‑based edit mechanism and the L‑EBPO reinforcement framework, allowing diffusion language models to delete and insert tokens during agent interactions, achieving near‑autoregressive accuracy (53.83 vs 55.74) and 1.64× higher BF16 throughput, plus an 8.6 % gain on SWE‑bench.

LLaDA2.2Levenshtein editingMoE
0 likes · 11 min read
LLaDA2.2 Released: Levenshtein Editing Enables Diffusion Language Models to Correct On-the-Fly
Machine Heart
Machine Heart
Jul 16, 2026 · Artificial Intelligence

Inkling: 975 B‑Parameter Open‑Weight Model from Thinking Machines Lab Targeting Customizable AI

Inkling, a 975‑billion‑parameter hybrid‑expert Transformer released by Thinking Machines Lab, offers fully open weights, multimodal capabilities across text, image, audio and video, controllable inference intensity, and extensive benchmark results, while also providing a smaller 276‑billion‑parameter variant and fine‑tuning support via the Tinker platform.

InklingLLMMoE
0 likes · 15 min read
Inkling: 975 B‑Parameter Open‑Weight Model from Thinking Machines Lab Targeting Customizable AI
Meituan Technology Team
Meituan Technology Team
Jul 1, 2026 · Artificial Intelligence

LongCat‑2.0: Training a Trillion‑Parameter Model on a Domestic 50k‑Card Cluster

Meituan’s LongCat‑2.0, a 1.6‑trillion‑parameter MoE model trained on a 50,000‑card domestic cluster, supports 1 M‑token context, uses Sparse Attention, zero‑compute experts and MOPD architecture, achieving over 1 T tokens/day throughput, 1.5× MFU efficiency, and top‑ranked scores on coding and agent benchmarks.

AI ModelAgentic CodingLongCat-2.0
0 likes · 10 min read
LongCat‑2.0: Training a Trillion‑Parameter Model on a Domestic 50k‑Card Cluster
Kuaishou Tech
Kuaishou Tech
Jun 11, 2026 · Artificial Intelligence

Keye-VL-2.0 Brings DeepSeek Sparse Attention to Multimodal AI – Report Released

Keye‑VL‑2.0, an open‑source MoE multimodal foundation model, tackles hour‑level video understanding and agentic intelligence by embedding DeepSeek Sparse Attention into a GQA‑based architecture, enabling near‑lossless 256 K token context, four‑stage pre‑training, diverse RL distillation techniques, and achieving state‑of‑the‑art results on long‑video benchmarks, with weights publicly released.

MoEPretrainingRL distillation
0 likes · 8 min read
Keye-VL-2.0 Brings DeepSeek Sparse Attention to Multimodal AI – Report Released
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 11, 2026 · Artificial Intelligence

Google’s 26B DiffusionGemma Model Delivers 1000+ Tokens/s – Runs on a 4090

DiffusionGemma, Google DeepMind’s 26B MoE model that generates 256‑token blocks via diffusion, achieves over 1000 tokens per second on H100/H200 GPUs, offers FP8 and NVFP4 quantized versions with near‑lossless accuracy, and can be deployed locally with vLLM Docker images, though it incurs higher first‑token latency and limited concurrency.

26B modelDiffusionGemmaFP8 quantization
0 likes · 10 min read
Google’s 26B DiffusionGemma Model Delivers 1000+ Tokens/s – Runs on a 4090
Su San Talks Tech
Su San Talks Tech
Jun 9, 2026 · Artificial Intelligence

Zero‑Cost Unlimited‑Token Access to Qwen 3.6: A Step‑by‑Step Guide

This article explains how developers can bypass token‑cost barriers by using iFlytek’s MaaS platform to obtain free, unlimited‑token access to the Qwen 3.6‑35B‑A3B model, details the model’s MoE architecture and benchmark performance, and provides a complete Java integration tutorial with code samples and practical use‑case suggestions.

AIAPIJava
0 likes · 16 min read
Zero‑Cost Unlimited‑Token Access to Qwen 3.6: A Step‑by‑Step Guide
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 1, 2026 · Artificial Intelligence

NVIDIA Unveils Nemotron 3 Ultra: The Largest US Open‑Source LLM Boosting Agent Capabilities

NVIDIA released Nemotron 3 Ultra, a 550 B‑parameter open‑source LLM with 55 B active MoE parameters, hybrid Mamba‑Transformer architecture, 1 M token context, and three core innovations that deliver superior MMLU, code, math scores and up to 5× throughput versus rivals, though weights are not yet public.

MambaMoENemotron 3 Ultra
0 likes · 8 min read
NVIDIA Unveils Nemotron 3 Ultra: The Largest US Open‑Source LLM Boosting Agent Capabilities
Old Zhang's AI Learning
Old Zhang's AI Learning
May 31, 2026 · Artificial Intelligence

Qwen3.6-35B-A3B NVFP4: A Stable, Highly Compressed Quantized Model

NVIDIA's NVFP4 quantization reduces Qwen3.6-35B-A3B's memory footprint by threefold with almost no accuracy loss, offers plug‑and‑play deployment via vLLM, and outperforms other 4‑bit formats on Hopper/Blackwell GPUs, making it a practical choice for production AI workloads.

MoENVFP4Qwen3.6-35B-A3B
0 likes · 13 min read
Qwen3.6-35B-A3B NVFP4: A Stable, Highly Compressed Quantized Model
Xiaomi Tech
Xiaomi Tech
May 30, 2026 · Artificial Intelligence

How Xiaomi’s MiMo V2.5 Achieves 99% API Price Cut with Full‑Stack Inference Optimizations

The MiMo‑V2.5 series combines Hybrid Sliding‑Window Attention, Mixture‑of‑Experts and multimodal support with a complete redesign of KVCache management, tiered caching, prefix‑tree logic and scheduling, compressing KVCache to about one‑seventh of full‑attention models and delivering up to 40% faster Prefill, 30% lower TTFT and dramatically reduced inference costs that enable a 99% API price reduction.

Hybrid SWAInference OptimizationKVCache
0 likes · 12 min read
How Xiaomi’s MiMo V2.5 Achieves 99% API Price Cut with Full‑Stack Inference Optimizations
Tencent Technical Engineering
Tencent Technical Engineering
May 24, 2026 · Artificial Intelligence

How Tsinghua & Tencent Mixed‑X Won the MLSys 2026 MoE Inference Challenge with a 4.1× Speedup

The Tsinghua‑Tencent Mixed‑X team captured the MLSys 2026 MoE inference optimization championship by analyzing NPU bottlenecks, redesigning data movement, applying expert‑level sharding, continuous DMA, PSUM batching, and an Agent‑based optimizer, achieving a 4.1× end‑to‑end speedup while preserving bit‑level output fidelity.

Agent optimizerInference OptimizationMLSys 2026
0 likes · 14 min read
How Tsinghua & Tencent Mixed‑X Won the MLSys 2026 MoE Inference Challenge with a 4.1× Speedup
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
May 16, 2026 · Artificial Intelligence

Token Superposition Training Accelerates LLM Pre‑training 2.5× Without Changing Architecture

Token Superposition Training (TST) speeds up large‑language‑model pre‑training by up to 2.5× without altering model architecture or compute budget, using a superposition phase that averages token embeddings into bags and predicts groups of tokens, followed by a standard recovery phase, as demonstrated on 10B‑parameter MoE and smaller models.

LLM PretrainingMCE lossMoE
0 likes · 10 min read
Token Superposition Training Accelerates LLM Pre‑training 2.5× Without Changing Architecture
SuanNi
SuanNi
May 12, 2026 · Artificial Intelligence

AntAngelMed: 6.1B‑Activated MoE Model Tops Three Medical Benchmarks

AntAngelMed, a 100‑billion‑parameter medical LLM using a 6.1 billion‑parameter MoE architecture, achieves performance comparable to a 40 billion‑parameter dense model, exceeds 200 tokens/s inference speed, and ranks first on HealthBench, MedAIBench and MedBench, with a three‑stage training pipeline and extensive efficiency optimizations.

HealthBenchMedAIBenchMedBench
0 likes · 6 min read
AntAngelMed: 6.1B‑Activated MoE Model Tops Three Medical Benchmarks
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
May 6, 2026 · Artificial Intelligence

Why DeepSeek‑V4’s MFU Drops: Parallel Strategies and Compute‑Communication Overlap

The article dissects DeepSeek‑V4’s shift from dense to MoE models, explains why MFU plummets despite sufficient expert dimensions, and details how a carefully designed GPU parallel strategy—combining DP, ZeRO‑1, PP, EP and the new Waved‑EP kernel—overlaps communication and computation to reclaim throughput on 8‑card NVLink nodes linked by InfiniBand.

GPU Distributed TrainingMoEPipeline Parallel
0 likes · 19 min read
Why DeepSeek‑V4’s MFU Drops: Parallel Strategies and Compute‑Communication Overlap
Architects' Tech Alliance
Architects' Tech Alliance
May 4, 2026 · Artificial Intelligence

DeepSeek‑V4 Inference Cost Showdown: NVIDIA H100 vs Ascend 950PR vs 910C

DeepSeek‑V4, a 1.6‑trillion‑parameter MoE model with mixed‑precision attention, is benchmarked on three accelerators—NVIDIA H100, Huawei Ascend 910C, and Ascend 950PR—showing that the 950PR delivers the lowest per‑token cost in both Prefill and Decode phases, while the H100 offers the highest raw performance at a far greater price.

FP8Huawei Ascend 950PRMoE
0 likes · 8 min read
DeepSeek‑V4 Inference Cost Showdown: NVIDIA H100 vs Ascend 950PR vs 910C
Architects' Tech Alliance
Architects' Tech Alliance
May 2, 2026 · Artificial Intelligence

Eight Chinese AI Chips Achieve Day‑Zero DeepSeek‑V4 Compatibility

The article explains how eight domestic AI chip makers—Huawei Ascend, Cambricon, HaiGuang, Moore Threads, Kunlun, Pingtouge, Muxi, and Tianshu—simultaneously completed full‑link compatibility, performance tuning, and stability verification for DeepSeek‑V4 on release day, detailing each vendor’s technical path, shared ecosystem breakthroughs, and the broader impact on the AI industry.

AI chipsDay0 adaptationFlagOS
0 likes · 11 min read
Eight Chinese AI Chips Achieve Day‑Zero DeepSeek‑V4 Compatibility
Architects' Tech Alliance
Architects' Tech Alliance
May 1, 2026 · Artificial Intelligence

How DeepSeek V4 Triggers a Global AI Price War with OpenAI

DeepSeek V4’s open‑source 1 M‑token MoE model delivers benchmark scores of MMLU 88.7, C‑Eval 92.1 and HumanEval 69.5, while its 4‑bit AWQ quantization, PagedAttention memory management and FlashAttention acceleration cut inference costs and latency, prompting rivals such as Anthropic, OpenAI, Baidu and Huawei to slash prices and boost efficiency in a fierce market battle.

AI EfficiencyMoEdeepseek-v4
0 likes · 9 min read
How DeepSeek V4 Triggers a Global AI Price War with OpenAI
DataFunTalk
DataFunTalk
Apr 28, 2026 · Artificial Intelligence

Manifold AI’s WorldScape 0.2 Tops WorldArena: How MoE Drives Superior Physics and 3D Understanding

Manifold AI’s WorldScape 0.2 achieved the highest overall score on the embodied world‑model benchmark WorldArena, outperforming giants like Google and Nvidia by excelling in comprehensive perception, physics compliance, and 3D accuracy while using only about 10 % of the parameters of competing models, thanks to a newly introduced MoE architecture.

Embodied AIMoEScaling Law
0 likes · 9 min read
Manifold AI’s WorldScape 0.2 Tops WorldArena: How MoE Drives Superior Physics and 3D Understanding
DeepHub IMBA
DeepHub IMBA
Apr 27, 2026 · Artificial Intelligence

DeepSeek‑V4 Deep Dive: Engineering Million‑Token Context Efficiency

The article provides a thorough technical analysis of DeepSeek‑V4, detailing how mixed sparse attention (CSA + HCA), manifold‑constrained hyper‑connections, the Muon optimizer, FP4 quantization, and a suite of infrastructure tricks enable stable training and inference with up to one‑million token contexts while achieving state‑of‑the‑art benchmark results.

CSAFP4 quantizationHCA
0 likes · 22 min read
DeepSeek‑V4 Deep Dive: Engineering Million‑Token Context Efficiency
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 25, 2026 · Artificial Intelligence

Why DeepSeek‑V4 Took Twice as Long: Inside the Training‑Stability Challenges and Engineering Hacks

The DeepSeek‑V4 technical report reveals that the model’s doubled training time stems from massive token and parameter scaling, severe training‑stability issues in MoE layers, and a suite of engineering solutions—including Anticipatory Routing, SwiGLU Clamping, specialist expert training, and a custom sandbox cluster—while also exposing high hallucination rates despite impressive benchmark performance.

Generative Reward ModelLLMMoE
0 likes · 12 min read
Why DeepSeek‑V4 Took Twice as Long: Inside the Training‑Stability Challenges and Engineering Hacks
Architects' Tech Alliance
Architects' Tech Alliance
Apr 24, 2026 · Artificial Intelligence

DeepSeek V4 Launches with 1M‑Token Context, Dual Versions and Native Chinese Chip Support

On April 24, 2026 DeepSeek released the V4 preview featuring two models—V4‑Pro with a 1.6 T‑parameter MoE architecture and V4‑Flash with 284 B parameters—both offering 1 million token context, up to 384 K output tokens, new step‑wise reasoning modes, and full native compatibility with Huawei Ascend and Cambricon chips, while delivering major efficiency gains and benchmark‑leading performance.

1M token contextCambriconDeepSeek
0 likes · 7 min read
DeepSeek V4 Launches with 1M‑Token Context, Dual Versions and Native Chinese Chip Support
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 23, 2026 · Artificial Intelligence

DeepSeek Quietly Open‑Sources TileKernels to Push GPU Performance to Its Limits

DeepSeek has released TileKernels, a GPU kernel library written in the TileLang DSL, that targets H100/H200/B200 GPUs and claims to approach hardware limits in compute intensity and memory bandwidth, offering MoE routing, FP8/FP4 quantization, and dual‑language PyTorch references for deep‑learning engineers.

FP8 quantizationGPU optimizationLLM training
0 likes · 9 min read
DeepSeek Quietly Open‑Sources TileKernels to Push GPU Performance to Its Limits
PaperAgent
PaperAgent
Apr 22, 2026 · Artificial Intelligence

Alibaba Unveils Four New Open‑Source Qwen3.6 Models: 27B Dense and 35B‑A3B MoE

Alibaba has added four new open‑source weight versions to its Qwen3.6 series, featuring the 27‑billion‑parameter dense multimodal model Qwen3.6‑27B and the 35‑billion‑parameter sparse expert model Qwen3.6‑35B‑A3B, both designed for stable, real‑world coding tasks and outperforming their Qwen3.5 predecessors.

AI agentsAlibabaDense Model
0 likes · 4 min read
Alibaba Unveils Four New Open‑Source Qwen3.6 Models: 27B Dense and 35B‑A3B MoE
PaperAgent
PaperAgent
Apr 21, 2026 · Artificial Intelligence

OpenMythos: Rebuilding Claude Mythos with Recursive Transformers and MoE

OpenMythos is an open‑source PyTorch reimplementation of Anthropic's Claude Mythos that uses a mixed‑expert routed recurrent Transformer, introduces Recursive Depth Transformers, Multi‑Latent Attention, and several stability mechanisms, and demonstrates parameter‑efficient scaling backed by empirical studies.

AI ArchitectureClaude MythosMoE
0 likes · 6 min read
OpenMythos: Rebuilding Claude Mythos with Recursive Transformers and MoE
HyperAI Super Neural
HyperAI Super Neural
Apr 21, 2026 · Artificial Intelligence

Qwen3.6-35B-A3B Boosts Agent Programming: 3B Activation Beats Gemma4-31B

Qwen3.6-35B-A3B, the first open‑source Qwen3.6 model, achieves markedly better scores than Qwen3.5‑35B‑A3B and Gemma4‑31B on Terminal‑Bench2.0, NL2Repo, and QwenClawBench, adds a thought‑process retention option, and is accessible via HyperAI’s ready‑to‑run notebook with free compute credits.

HyperAIMoEQwen3.6
0 likes · 4 min read
Qwen3.6-35B-A3B Boosts Agent Programming: 3B Activation Beats Gemma4-31B
Lao Guo's Learning Space
Lao Guo's Learning Space
Apr 8, 2026 · Artificial Intelligence

2026 Qwen Model Comparison: Choose the Right Qwen for Your Mac Studio

An in‑depth 2026 comparative review of Alibaba’s Qwen series (Qwen2.5, Qwen3, Qwen3.5) evaluates architecture, performance, speed and VRAM usage on Mac Studio, ranks each variant, and provides concrete model‑selection guidance for different memory configurations, highlighting the MoE‑based Qwen3.5 as the optimal choice.

AI performanceMac StudioMoE
0 likes · 9 min read
2026 Qwen Model Comparison: Choose the Right Qwen for Your Mac Studio
ShiZhen AI
ShiZhen AI
Mar 17, 2026 · Artificial Intelligence

Kimi’s Attention Residuals Swap a Decade-Old Residual Trick for 1.25× Faster 48B MoE

The Kimi team introduces Attention Residuals, a softmax‑based replacement for the uniform residual connections used in Transformers for a decade, enabling selective aggregation of layer histories, reducing hidden‑state growth, and achieving a 1.25× compute‑efficiency gain on a 48‑billion‑parameter MoE model with less than 2% inference latency increase.

Attention ResidualsMoEResidual Connection
0 likes · 10 min read
Kimi’s Attention Residuals Swap a Decade-Old Residual Trick for 1.25× Faster 48B MoE
Old Zhang's AI Learning
Old Zhang's AI Learning
Mar 13, 2026 · Artificial Intelligence

Nvidia’s New OpenClaw‑Optimized Model Cracks Top‑5 on PinchBench – Free to Use

Nvidia’s open‑source Nemotron‑3‑Super model achieves an 85.6% success rate on the PinchBench OpenClaw benchmark, ranking in the top five (the only open‑source entry), and the article explains its architecture, quantization, training pipeline, performance numbers, usage options, and practical limitations.

AI coding agentMoENVFP4
0 likes · 10 min read
Nvidia’s New OpenClaw‑Optimized Model Cracks Top‑5 on PinchBench – Free to Use
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 26, 2026 · Artificial Intelligence

Ultimate Guide to Local Deployment of Qwen3.5 Models (27B‑397B)

This guide reviews the Qwen3.5 model lineup, explains mixed‑inference and MoE architecture, presents benchmark comparisons with GPT‑5.2, Claude 4.5 and Gemini‑3 Pro, evaluates 4‑bit and 3‑bit quantization loss, outlines hardware requirements, and provides step‑by‑step deployment options using llama.cpp or llama‑server.

MoEQwen3.5inference
0 likes · 14 min read
Ultimate Guide to Local Deployment of Qwen3.5 Models (27B‑397B)
Baobao Algorithm Notes
Baobao Algorithm Notes
Feb 25, 2026 · Artificial Intelligence

Exploring Qwen 3.5: Small‑Scale MoE Models, Architecture, and Deployment Guides

This article reviews the three open‑source Qwen 3.5 models—including a 35B MoE, a 122B MoE, and a 27B dense version—detailing their parameter layouts, core attention designs, context length, inference performance, hardware requirements, and provides step‑by‑step code examples for loading them with Hugging Face Transformers and vLLM.

AIMoEQwen
0 likes · 10 min read
Exploring Qwen 3.5: Small‑Scale MoE Models, Architecture, and Deployment Guides
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 19, 2026 · Artificial Intelligence

Inside GLM-5: Training Techniques, Architecture Innovations, and Benchmark Performance

The article dissects GLM-5’s 744B‑parameter MoE design, 28.5 T token training corpus, novel Muon Split and MLA‑256 optimizations, DSA sparse attention, a fully asynchronous RL pipeline, extensive domestic chip adaptation, and benchmark results that place it on par with Claude Opus 4.5 and ahead of Gemini 3 Pro.

AI ArchitectureDSAGLM-5
0 likes · 13 min read
Inside GLM-5: Training Techniques, Architecture Innovations, and Benchmark Performance
Node.js Tech Stack
Node.js Tech Stack
Feb 16, 2026 · Artificial Intelligence

Qwen 3.5 Launch: 17B Active Parameters Take on GPT‑5.2

Qwen 3.5, an open‑source 397B‑parameter model that activates only 17B parameters, uses a hybrid MoE‑Gated Delta architecture, offers native multimodal support and a default chain‑of‑thought mode, and achieves benchmark scores comparable to GPT‑5.2, Claude 4.5 Opus and Gemini 3 Pro across code, math, agent and vision tasks.

AI ModelMoEQwen 3.5
0 likes · 9 min read
Qwen 3.5 Launch: 17B Active Parameters Take on GPT‑5.2
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 9, 2026 · Artificial Intelligence

GLM-5 Emerges First, Built on DeepSeek Tech, Triggering a 40% Stock Surge

An anonymous OpenRouter model dubbed "Pony Alpha" was verified as the new 745B‑parameter GLM-5, which reuses DeepSeek‑V3 architecture, supports sparse attention and multi‑token prediction, and has already caused a near‑40% jump in Zhipu AI’s stock while hinting at upcoming integration into the Transformers library.

DeepSeekGLM-5MoE
0 likes · 3 min read
GLM-5 Emerges First, Built on DeepSeek Tech, Triggering a 40% Stock Surge
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 3, 2026 · Artificial Intelligence

Step‑3.5‑Flash: Lightning‑Fast Inference with 196B Params, Only 11B Active (vLLM)

Step‑3.5‑Flash, a 196‑billion‑parameter open‑source LLM that activates only 11 B per token via a Mixture‑of‑Experts design, delivers 3‑plus‑times faster inference, matches top‑tier closed‑source models on SWE‑bench and other benchmarks, supports 256 K context, runs on consumer‑grade hardware, and is already integrated into vLLM, SGLang, and Claude Code, though it has known token‑efficiency and domain‑stability limitations.

LLM benchmarkMoEMulti-Token Prediction
0 likes · 11 min read
Step‑3.5‑Flash: Lightning‑Fast Inference with 196B Params, Only 11B Active (vLLM)
Alibaba Cloud Developer
Alibaba Cloud Developer
Jan 26, 2026 · Artificial Intelligence

How We Scaled a 3.5B MoE LLM for Real‑Time Search Relevance

This article details the engineering challenges and solutions for deploying a 3.5 billion‑parameter MoE LLM in Taobao's search relevance pipeline, covering large‑batch scheduling, dynamic load balancing, intra‑batch KV‑Cache reuse, and MoE kernel tuning to meet sub‑second latency requirements.

Inference OptimizationKV CacheLLM
0 likes · 15 min read
How We Scaled a 3.5B MoE LLM for Real‑Time Search Relevance
Data Party THU
Data Party THU
Jan 13, 2026 · Artificial Intelligence

How Engram’s ‘Lookup‑Compute Separation’ Boosts LLM Performance

DeepSeek’s newly open‑sourced Engram module introduces a scalable lookup‑based memory that separates knowledge retrieval from computation, enabling O(1) deterministic access and significantly improving large language model performance on knowledge‑heavy, reasoning, code, and math tasks without extra FLOPs.

@LookupLLMMemory Architecture
0 likes · 10 min read
How Engram’s ‘Lookup‑Compute Separation’ Boosts LLM Performance
AI Insight Log
AI Insight Log
Dec 18, 2025 · Artificial Intelligence

Xiaomi’s New MiMo‑V2‑Flash LLM Rivals DeepSeek‑V3.2 and Near‑GPT‑5 High

Xiaomi’s MiMo‑V2‑Flash, a 309B‑parameter MoE LLM with only 15B active weights, uses Hybrid SWA, Multi‑Token Prediction and Multi‑Teacher On‑Policy Distillation to cut KV‑cache by six times, boost inference speed 2.6×, and achieve performance comparable to DeepSeek‑V3.2, Kimi‑K2 and near‑GPT‑5 High, including a 73.4% SWE‑Bench code‑agent score.

Hybrid SWAMOPDMTP
0 likes · 7 min read
Xiaomi’s New MiMo‑V2‑Flash LLM Rivals DeepSeek‑V3.2 and Near‑GPT‑5 High
Architect
Architect
Dec 15, 2025 · Artificial Intelligence

Demystifying LLM Architecture: From Transformers to Modern MoE Designs

This comprehensive guide explains the fundamentals of large language model (LLM) architectures, covering the original Transformer, tokenization, embeddings, positional encoding, attention mechanisms, feed‑forward networks, layer stacking, a step‑by‑step translation example, and the latest open‑source and hybrid LLM designs shaping the field.

AttentionEmbeddingLLM
0 likes · 41 min read
Demystifying LLM Architecture: From Transformers to Modern MoE Designs
Architects' Tech Alliance
Architects' Tech Alliance
Oct 24, 2025 · Artificial Intelligence

How xPU Scale‑Up Networks Are Redefining AI Training Efficiency

As AI models grow to massive scales, the demand for ultra‑high‑performance, low‑latency networking in xPU clusters intensifies, prompting a shift from dense to MoE architectures and driving the evolution of Scale‑up networks, where Alibaba Cloud’s UPN design tackles bandwidth, cost, and reliability challenges.

AIMoEUltraPerformanceNetwork
0 likes · 13 min read
How xPU Scale‑Up Networks Are Redefining AI Training Efficiency
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Sep 25, 2025 · Artificial Intelligence

Unlocking Trillion‑Parameter MoE Models: Expert Parallelism and Alibaba Cloud PAI‑EAS Deployment Guide

This article explains the opportunities and challenges of Mixture of Experts (MoE) models, introduces expert parallelism as a solution to scaling and deployment bottlenecks, and provides a step‑by‑step guide for deploying MoE models with Alibaba Cloud PAI‑EAS, including configuration tips and code examples.

AI model deploymentExpert ParallelismMoE
0 likes · 11 min read
Unlocking Trillion‑Parameter MoE Models: Expert Parallelism and Alibaba Cloud PAI‑EAS Deployment Guide
AntTech
AntTech
Sep 14, 2025 · Artificial Intelligence

Ring-mini-2.0: How a 16B MoE Model Delivers 128K Context and 500+ Tokens/s

Ring-mini-2.0 is a high‑performance inference MoE model that activates only 1.4 B parameters out of 16 B total, achieving dense‑model quality below 10 B while supporting 128 K context length and ultra‑fast generation speeds of over 300 tokens/s.

AIInference OptimizationMoE
0 likes · 4 min read
Ring-mini-2.0: How a 16B MoE Model Delivers 128K Context and 500+ Tokens/s
AntTech
AntTech
Sep 13, 2025 · Artificial Intelligence

LLaDA‑MoE: The First Native MoE Diffusion Language Model Shattering Autoregressive Limits

Ant Group and Renmin University unveiled LLaDA‑MoE, the industry’s first native MoE‑based diffusion language model trained on 20 TB of data, achieving performance comparable to Qwen2.5 while delivering several‑fold faster inference, and the model will be fully open‑sourced to accelerate global AI research.

AI researchLLaDA-MoEMoE
0 likes · 6 min read
LLaDA‑MoE: The First Native MoE Diffusion Language Model Shattering Autoregressive Limits
AntTech
AntTech
Sep 11, 2025 · Artificial Intelligence

Ling-mini-2.0: How a 16B MoE Model Achieves Dense-Level Performance with Only 1.4B Active Parameters

Ling-mini-2.0, an open-source 16 B MoE language model that activates only 1.4 B parameters, achieves dense-level performance with 7× efficiency, generates over 300 tokens / s, and introduces the first FP8 mixed-precision training suite, offering multiple pre-training checkpoints for the AI community.

FP8 trainingMoEefficient inference
0 likes · 6 min read
Ling-mini-2.0: How a 16B MoE Model Achieves Dense-Level Performance with Only 1.4B Active Parameters
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Sep 4, 2025 · Artificial Intelligence

Unlocking MoE Model Power: Baidu’s Baige 5.0 AI Platform’s FP8 and Distributed Innovations

Baidu’s Baige 5.0 AI Computing Platform introduces FP8 mixed‑precision training, MoE‑aware distributed strategies, adaptive parallelism, and a three‑tier KV‑Cache, delivering over 30% training speedup and 50% inference throughput gains while keeping token latency under half a second for large‑scale models.

AIFP8MoE
0 likes · 16 min read
Unlocking MoE Model Power: Baidu’s Baige 5.0 AI Platform’s FP8 and Distributed Innovations
Data Party THU
Data Party THU
Aug 11, 2025 · Artificial Intelligence

What Sets the Latest LLMs Apart? A Deep Dive into V3, OLMo, Gemma, Mistral, Llama 4 and More

This article systematically compares the architectures of recent large language models—including DeepSeek V3/R1, OLMo 2, Gemma 3, Mistral Small 3.1, Llama 4, Qwen 3, SmolLM 3 and Kimi 2—highlighting innovations such as MLA, MoE, post‑norm, sliding‑window attention, NoPE and optimizer choices, with diagrams and code examples to illustrate their impact on efficiency and performance.

LLMMLAMoE
0 likes · 12 min read
What Sets the Latest LLMs Apart? A Deep Dive into V3, OLMo, Gemma, Mistral, Llama 4 and More
AI Algorithm Path
AI Algorithm Path
Jul 29, 2025 · Artificial Intelligence

Why GLM‑4.5 Sets a New Benchmark for Open‑Source Large Language Models

GLM‑4.5 and its lightweight Air variant, featuring a deep‑layered MoE design, grouped‑query attention, and dual inference modes, achieve third‑place overall on 12 hard‑core benchmarks, excel in web‑browsing and tool‑calling with a 90.6 % success rate, and introduce novel training tricks such as the Muon optimizer and Slime RL framework.

AIGLM-4.5MoE
0 likes · 8 min read
Why GLM‑4.5 Sets a New Benchmark for Open‑Source Large Language Models
Tech Freedom Circle
Tech Freedom Circle
Jul 17, 2025 · Artificial Intelligence

DeepSeek V3 Architecture Deep Dive: MoE, MLA, DualPipe, FP8 Mixed Precision & Multi‑Token Prediction

This article provides a detailed technical analysis of DeepSeek‑V3, covering its MOE architecture, the novel Multi‑head Latent Attention (MLA) mechanism, the DualPipe pipeline‑parallel algorithm, mixed‑precision FP8 training, and the Multi‑Token Prediction (MTP) inference improvements that together boost performance and efficiency.

DeepSeekDualPipeFP8
0 likes · 44 min read
DeepSeek V3 Architecture Deep Dive: MoE, MLA, DualPipe, FP8 Mixed Precision & Multi‑Token Prediction
21CTO
21CTO
Jul 1, 2025 · Artificial Intelligence

OpenAI CEO Warns: Don’t Blindly Trust AI – Insights from New Open‑Source Models

Sam Altman cautions against over‑reliance on ChatGPT, while Germany blocks DeepSeek for GDPR violations, Tencent unveils its MoE‑based Hunyuan‑A13B model, and Google releases a Python client for Data Commons, highlighting both AI risks and rapid open‑source advancements.

AI safetyData CommonsMoE
0 likes · 9 min read
OpenAI CEO Warns: Don’t Blindly Trust AI – Insights from New Open‑Source Models
DataFunTalk
DataFunTalk
Jun 30, 2025 · Artificial Intelligence

Wenxin 4.5 Series: Open‑Source Multimodal MoE Models and FastDeploy Guide

The Wenxin 4.5 series introduces ten open‑source models—including large‑scale MoE and dense variants—featuring a novel multimodal heterogeneous architecture, high training efficiency, SOTA benchmark performance, and comprehensive toolkits (ERNIEKit, FastDeploy) for fine‑tuning and multi‑hardware deployment.

ERNIEKitFastDeployMoE
0 likes · 8 min read
Wenxin 4.5 Series: Open‑Source Multimodal MoE Models and FastDeploy Guide
Smart Era Software Development
Smart Era Software Development
May 30, 2025 · Artificial Intelligence

How Tencent’s TRMT Boosted DeepSeek’s Communication: A Chinese Open‑Source Success

Tencent’s Star‑Network team partnered with DeepSeek to open‑source the DeepEP communication library, then used its self‑developed TRMT stack to overcome RoCE limitations, achieving up to 100% speedup on RoCEv2 and 30% on InfiniBand, cutting training costs and inference latency for large MoE models.

AI TrainingDeepEPDeepSeek
0 likes · 8 min read
How Tencent’s TRMT Boosted DeepSeek’s Communication: A Chinese Open‑Source Success
AI Algorithm Path
AI Algorithm Path
May 9, 2025 · Artificial Intelligence

A Visual Guide to Mixture of Experts (MoE) Architecture in Large Language Models

This article explains the Mixture of Experts (MoE) technique used in modern LLMs, detailing its core components—experts and router—comparing dense and sparse layers, describing load‑balancing, expert capacity, and routing strategies, and showcasing real‑world examples such as Switch Transformer, Vision‑MoE, and Mixtral 8x7B.

Expert CapacityLLMMixture of Experts
0 likes · 15 min read
A Visual Guide to Mixture of Experts (MoE) Architecture in Large Language Models