DeepSeek‑R1, the first mainstream large language model to pass peer review in Nature, was trained for $294,000 using 648 H800 GPUs, and its RL‑enhanced version, DeepSeek‑R1‑Zero, achieved up to 86.7% pass@1 on AIME 2024, outperforming human averages across math, coding, and science tasks.
DeepSeek’s three‑model LLM lineup—V3, R1‑Zero and R1—delivers high performance while cutting training expenses to under $600 k, a fraction of the $0.6‑1 B typical for comparable models, signaling a major shift in China’s AI compute demand and supply chain dynamics.
The article analyzes DeepSeek’s latest V3 conversational model and R1 inference model, detailing their MoE architecture, training on H800 GPUs costing about $558 k, comparing compute expenses to Meta’s Llama 3.1, and showing that their API pricing is roughly one‑tenth of GPT‑4o for dialogue and one‑twentieth of OpenAI o1 for inference.
Arm CEO Rene Haas dismisses DeepSeek’s claimed $5.6 million training cost as a rumor, while the Chinese startup’s low‑cost, high‑performance models spark debate over AI development economics, geopolitics, and looming government bans worldwide.
This article provides a comprehensive analysis of DeepSeek's large‑language‑model technology, covering the company's background, model capabilities, remarkably low training and inference costs, and the core architectural and algorithmic innovations such as MoE, MLA attention, FP8 mixed‑precision, and the DualPipe pipeline that enable efficient large‑scale AI deployment.
The article provides a detailed analysis of DeepSeek’s model performance across language, reasoning, and code generation benchmarks, its cost‑effective training methods, novel architecture innovations, the team’s expertise, and the broader impact these factors may have on accelerating AI innovation and reshaping industry competition.
This article derives DeepSeek V3's training Model FLOPs Utilization (MFU) using publicly available data, showing an MFU of roughly 37%—about a 60% improvement over V2—and provides detailed formulas, parameter settings, and a reproducible Python script.
