Tagged articles
MLA
18 articles · Page 1 of 1
This article details how a custom C inference engine runs the 744B-parameter GLM 5.2 MoE model on 25GB RAM by quantizing routed experts to int4, streaming them from disk via pread, compressing KV cache 57x with Multi-head Latent Attention, and employing a five-tier memory hierarchy with speculative decoding.
AVX2 kernelsC inference engineGLM-5.2
0 likes · 48 min read

This week’s tech roundup covers Anthropic’s Claude 4.8 launch with higher honesty and parallel agent support, OpenAI’s GPT‑5.5 performance drop, Nvidia CEO joining Tsinghua, AI wealth hotspots in Beijing and San Francisco, emerging AI‑driven design language MLA, EverMind’s memory‑centric agents, three‑bit quantization enabling 600 B‑parameter models on phones, and new open‑source AI‑agent platforms such as PilotDeck.
AI agentsAI industryClaude 4.8
0 likes · 27 min read

This article presents GQLA, a single‑author variant of MLA that eliminates three hardware‑related drawbacks of MLA, demonstrates how it achieves balanced compute‑memory performance on both high‑end H100 and more modest H20 GPUs, and details conversion methods (TransGQLA) and sparse extensions with concrete benchmark results.
AttentionGQLALLM
0 likes · 16 min read

Ant’s newly released Ling‑2.6‑1T, a trillion‑parameter LLM, combines a hybrid MLA‑plus‑Linear Attention architecture to deliver 256K context, ultra‑low token cost and millisecond‑level latency, achieving GPT‑5.4‑level performance on multiple benchmarks while being open‑sourced for developers.
Ant AIFast ThinkingLLM benchmark
0 likes · 10 min read

The article analyses DeepSeek V4’s technical report, revealing repeated “encounters” between DeepSeek and Kimi—shared MLA attention, Muon optimizer, and divergent long‑context strategies—while highlighting their open‑source releases, hardware adaptations, and ecosystem impact that dramatically lower deployment costs for Chinese AI.
AIDeepSeekKimi
0 likes · 10 min read

This article reviews and compares the most important attention variants used in modern large language models—including multi‑head attention, grouped‑query attention, multi‑head latent attention, sparse and sliding‑window attention, gated attention, and hybrid designs—detailing their motivations, memory trade‑offs, example architectures, and experimental findings.
GQALLMMHA
0 likes · 29 min read

This article walks through the DeepSeek‑V3.2‑Exp inference codebase, detailing its MoE architecture, Multi‑Head Latent Attention, FP8 quantization, custom CUDA kernels, and 8‑GPU NCCL‑based distributed execution from initialization through prefill and decode stages.
CUDAFP8 quantizationLLM
0 likes · 9 min read

This article systematically compares the architectures of recent large language models—including DeepSeek V3/R1, OLMo 2, Gemma 3, Mistral Small 3.1, Llama 4, Qwen 3, SmolLM 3 and Kimi 2—highlighting innovations such as MLA, MoE, post‑norm, sliding‑window attention, NoPE and optimizer choices, with diagrams and code examples to illustrate their impact on efficiency and performance.
LLMMLAMoE
0 likes · 12 min read

This article provides a detailed technical analysis of DeepSeek‑V3, covering its MOE architecture, the novel Multi‑head Latent Attention (MLA) mechanism, the DualPipe pipeline‑parallel algorithm, mixed‑precision FP8 training, and the Multi‑Token Prediction (MTP) inference improvements that together boost performance and efficiency.
DeepSeekDualPipeFP8
0 likes · 44 min read

PaddlePaddle 3.0 introduces a full‑stack inference engine that supports FP8, INT8, and 4‑bit quantization for popular LLMs such as DeepSeek V3/R1, delivers up to 2× token throughput on a single H800 GPU, and provides detailed deployment scripts for single‑node and multi‑node setups, including MTP speculative decoding and SageAttention for long‑sequence acceleration.
DockerInference OptimizationMLA
0 likes · 13 min read


ArchitectMar 10, 2025 · Artificial Intelligence
This article analyzes DeepSeek’s latest large‑model breakthroughs, covering the MLA attention compression, GRPO alignment algorithm, MoE load‑balancing redesign, multi‑stage training pipelines, reinforcement‑learning tricks, and performance comparisons with GPT‑4o‑Mini and Llama 3.1, highlighting both strengths and remaining challenges.
AI TrainingDeepSeekGRPO
0 likes · 19 min read

DeepSeek’s OpenSourceWeek introduced FlashMLA, a GPU‑optimized MLA decoding kernel for Hopper GPUs that leverages FlashAttention and CUTLASS to dramatically improve large‑model inference performance, with early adoption showing up to 30% higher compute utilization and doubled speed in some scenarios.
DeepSeekFlashMLAGPU
0 likes · 3 min read

The article provides an in‑depth overview of DeepSeek’s large language model, detailing its mixture‑of‑experts and Transformer foundations, novel attention mechanisms, load‑balancing, multi‑token prediction, FP8 mixed‑precision training, and various training regimes such as knowledge distillation and reinforcement learning.
DeepSeekFP8MLA
0 likes · 18 min read

This tutorial walks users through installing FlashMLA, integrating it with the vLLM framework, downloading the DeepSeek‑V2‑Lite‑Chat model, benchmarking various MLA implementations, and running a local inference demo that shows FlashMLA’s speed advantage on long‑sequence generation.
DeepSeekFlashMLAInferenceOptimization
0 likes · 16 min read

Flash-MLA is an open‑source GPU kernel optimized for Nvidia Hopper GPUs that compresses the KV cache of multi‑head attention, cutting memory usage by up to 93.3% and delivering 580 TFLOPS compute, thereby dramatically accelerating large‑language‑model inference while lowering cost.
DeepSeekFlash-MLAGPU optimization
0 likes · 8 min read

This article presents a theoretical and experimental analysis of converting Group Query Attention (GQA) models to Multi‑Head Linear Attention (MLA) using the TransMLA method, demonstrating superior expressiveness and performance on DeepSeek‑based large language models while keeping KV‑Cache costs unchanged.
AttentionDeepSeekGQA
0 likes · 11 min read

This article analyzes DeepSeek's V3 and R1 large language models, detailing their low‑cost Mixture‑of‑Experts architecture, Multi‑Head Latent Attention redesign, distributed training optimizations, and reasoning‑focused innovations that together challenge traditional GPU/NPU compute demands.
AI inferenceDeepSeekMLA
0 likes · 15 min read

From the perceptron’s inception in 1958 to modern Transformer-based models like GPT, this article traces the evolution of deep learning, explaining foundational architectures such as DNNs, CNNs, RNNs, LSTMs, attention mechanisms, and recent innovations like DeepSeek’s MLA, highlighting their principles and impact.
AttentionGPTMLA
0 likes · 19 min read
