Open‑Sora 2.0 is an open‑source 11B‑parameter video generation model that matches commercial SOTA performance while being trained on 224 GPUs for just $200,000, thanks to a 3D auto‑encoder, MMDiT architecture, aggressive data filtering, low‑resolution pre‑training, and highly optimized parallel training techniques.
This article analyzes DeepSeek's V3 and R1 large language models, detailing their low‑cost Mixture‑of‑Experts architecture, Multi‑Head Latent Attention redesign, distributed training optimizations, and reasoning‑focused innovations that together challenge traditional GPU/NPU compute demands.
DeepSeek, a Chinese AI model, surged to the top of both China and US Apple App Store free‑app charts, outpacing ChatGPT and other major generative AI services, while boasting dramatically lower training costs and an open‑source approach that has sparked worldwide attention.
This article explains how researchers reproduced the full ChatGPT training pipeline—including supervised fine‑tuning, reward‑model training, and RLHF—using the open‑source Colossal‑AI system, dramatically reducing GPU memory and hardware requirements while providing ready‑to‑run code and performance benchmarks.
