A 30‑person lab trained a 749B‑parameter Agent model called Macaron‑V1‑Preview using fewer than 300 GPUs, achieving less than 1% of the compute cost of comparable models while matching state‑of‑the‑art performance on real‑world Agent benchmarks such as LivingBench, VitaBench, A2UI and PinchBench.
MindLab released the 749B‑parameter Macaron‑V1‑Preview, a model engineered for deep Agent‑Harness post‑training that was trained on fewer than 300 GPUs at less than 1% of the compute cost of peer models and achieves SOTA results on multiple Agent‑centric benchmarks such as LivingBench, VitaBench and PinchBench.
The article presents the Attention Residuals architecture, explains how it replaces uniform residual addition with learned attention‑based aggregation, details full and block variants, engineering tricks for distributed training, and shows extensive scaling‑law experiments where the new design consistently improves validation loss and training efficiency across model sizes.
SANA 1.5 introduces an efficient model‑growth pipeline, depth‑pruning, and inference‑time scaling that reuse a 1.6 B‑parameter foundation to train a 4.8 B model with 8× lower memory, 60 % less training time, and GenEval scores that rival or surpass much larger diffusion models.
Microsoft open‑sourced DeepSpeed‑Chat, a toolkit that streamlines the end‑to‑end training and inference of ChatGPT‑like large language models using RLHF, delivering up to fifteen‑fold speedups and dramatically lower costs, even on a single GPU.
This article surveys the evolution of pre‑trained models, covering the origins of transfer and self‑supervised learning, the rise of transformer‑based PTMs such as BERT and GPT, efficient architecture designs, multimodal and multilingual extensions, theoretical analyses, and future research directions for scalable and robust AI systems.
