BigMac introduces a dependency‑safe nested pipeline that preserves LLM compute efficiency while reducing encoder and generator activation memory to O(1), delivering 1.08‑1.9× speedup and stable memory usage for large‑scale multimodal model training.
OpenAI unveiled Dreaming V3, a new memory architecture that lets ChatGPT silently replay and consolidate daily conversations, achieving 82.8% context recall, 71.3% preference compliance, five‑fold compute savings, and free access for billions while offering a transparent memory‑summary interface.
Anthropic’s decision to block third‑party Agent frameworks from Claude’s subscription model exposes unsustainable token pricing, highlights massive compute waste caused by poor context handling, and argues that the real solution lies in smarter, more efficient agent design rather than cheaper tokens.
SkillCraft, a new benchmark from Oxford and partner institutions, evaluates whether AI agents can autonomously combine basic tools into reusable skills, revealing that stronger models dramatically improve task success rates while slashing compute consumption by up to 80%, and exposing the limits of hierarchical skill nesting and cross‑model skill sharing.
The Kimi team introduces Attention Residuals, a softmax‑based replacement for the uniform residual connections used in Transformers for a decade, enabling selective aggregation of layer histories, reducing hidden‑state growth, and achieving a 1.25× compute‑efficiency gain on a 48‑billion‑parameter MoE model with less than 2% inference latency increase.
The paper introduces JTok and its dynamic variant JTok‑M, a token‑indexed parameter scaling method that decouples model capacity from compute, achieving up to 35% compute reduction while delivering consistent performance gains across a wide range of downstream tasks and model sizes.
AI success hinges on the interplay of compute, algorithms, and data; the article breaks down each component, highlights common pitfalls such as poor data quality or insufficient compute, and offers practical strategies—from data governance to algorithm system design and cost‑effective compute—to turn AI from usable to truly useful.
This article explains how scaling laws can answer common LLM development questions—such as the data required for a 10B model, the model size achievable with 1 TB of data, and the optimal compute‑data‑model trade‑off for a fixed GPU budget—by presenting core formulas, practical derivations, and insights from OpenAI, DeepMind and Google.
This article provides an extensive retrospective and forward‑looking analysis of the evolution of click‑through‑rate prediction technologies in online advertising, covering shallow‑learning era challenges, the rise of industrial‑scale deep learning, system‑level innovations such as recall, coarse‑ranking, fine‑ranking, bidding, and the emerging co‑design of algorithms, compute, and architecture.
