Tagged articles

SMELT

2 articles · Page 1 of 1

Sep 19, 2026 · Artificial Intelligence

Looped Transformers: Recurrent Depth, Adaptive Recursion, Reasoning vs Memory

This article dissects the looped transformer architecture behind GPT-6 Astra, explaining recurrent depth, adaptive recursion mechanisms from Universal Transformer to Mixture-of-Recursions, and research showing recurrence improves multi-step reasoning without increasing knowledge capacity, while reducing training compute by 6.8–18% at scale.

Adaptive ComputationGPT-6 AstraLooped Transformer

0 likes · 12 min read

Looped Transformers: Recurrent Depth, Adaptive Recursion, Reasoning vs Memory

Machine Heart

Sep 8, 2026 · Artificial Intelligence

SMELT: Looped Transformers Outperform Baselines Under Fair Budget Matching

The SMELT framework from Tsinghua and ByteDance Seed fairly compares Looped Transformers against baselines by matching compute, parameters, and KV cache, finding that looping the middle 50% of layers twice with a larger depth-width ratio consistently reduces validation loss across scales, saves 6.8–18% training compute, and yields downstream gains beyond loss reduction.

Budget MatchingLLM ArchitectureLooped Transformer

0 likes · 12 min read

SMELT: Looped Transformers Outperform Baselines Under Fair Budget Matching