Open‑Sora 2.0 is an open‑source 11B‑parameter video generation model that matches commercial SOTA performance while being trained on 224 GPUs for just $200,000, thanks to a 3D auto‑encoder, MMDiT architecture, aggressive data filtering, low‑resolution pre‑training, and highly optimized parallel training techniques.
This article analyzes DeepSeek's V3 and R1 large language models, detailing their low‑cost Mixture‑of‑Experts architecture, Multi‑Head Latent Attention redesign, distributed training optimizations, and reasoning‑focused innovations that together challenge traditional GPU/NPU compute demands.
The article dissects the s1 paper that claims a sub‑$50 cloud budget can produce a reasoning model rivaling DeepSeek R1 and OpenAI o1, detailing the curated s1K dataset, the budget‑forcing inference technique, the 26‑minute fine‑tuning on Qwen2.5‑32B, performance gaps on AIME and MATH benchmarks, and the misconceptions surrounding cost and "distillation".
Researchers from Stanford and Washington trained the s1 inference model in just 26 minutes using under $50 of cloud credits, achieving performance comparable to OpenAI's o1 and DeepSeek's R1 by building a curated 1,000‑sample dataset and a budget‑enforced test‑time scaling algorithm.
DeepSeek, a Chinese AI model, surged to the top of both China and US Apple App Store free‑app charts, outpacing ChatGPT and other major generative AI services, while boasting dramatically lower training costs and an open‑source approach that has sparked worldwide attention.
This article explains how researchers reproduced the full ChatGPT training pipeline—including supervised fine‑tuning, reward‑model training, and RLHF—using the open‑source Colossal‑AI system, dramatically reducing GPU memory and hardware requirements while providing ready‑to‑run code and performance benchmarks.
