DeepSeek Model Innovations: Architecture, Training Methods, and Performance Evaluation

DeepSeek introduces several core architectural innovations. The Multi‑Head Latent Attention (MLA) reduces KV cache size by 93.3%, enabling more efficient inference. Future extensions such as Quantized‑MLA (QMLA) or Compressed‑MLA (CMLA) are anticipated.

The Group Relative Policy Optimization (GRPO) algorithm improves alignment without requiring a same‑scale evaluator model, using group‑wise scoring to estimate baselines. Reward modeling combines accuracy, format, and language consistency rewards, with specific mechanisms for verifying mathematical answers and code generation.

DeepSeek's MoE strategy replaces large experts with many tiny ones, adding shared experts and eliminating auxiliary‑loss load balancing. Dynamic bias adjustments ensure balanced routing without degrading specialist performance, resulting in more stable training and better utilization of expert capacity.

Training innovations are presented in three stages. First, a traditional transformer‑based pre‑training creates a base model. Second, supervised fine‑tuning (SFT) refines instruction following. Third, preference tuning aligns model behavior with human preferences. DeepSeek‑R1 also incorporates reinforcement learning, using a mid‑stage reasoning model (R1‑Zero) trained solely via RL to generate high‑quality chain‑of‑thought data for SFT.

R1‑Zero demonstrates strong reasoning without any supervised data, while R1 adds supervised fine‑tuning and safety improvements to address readability and language mixing issues. Cold‑start data, consisting of a few long‑chain reasoning examples, mitigates early RL instability.

Knowledge distillation transfers the capabilities of the large DeepSeek‑R1 model to smaller variants (e.g., 1.5B‑parameter models), preserving reasoning steps and reducing computational costs.

Performance evaluations show DeepSeek‑R1 matching or surpassing GPT‑4o‑Mini in intelligence and consistency, and outperforming Llama 3.1 in problem‑solving and safety metrics, though some harmful content generation remains a concern.

The article concludes with a summary of model versions (DeepSeek‑R1‑Zero and DeepSeek‑R1), detailed comparisons, and references to additional resources.