BestHub
Sign in
Tagged articles

Looped Transformer

4 articles · Page 1 of 1
PaperAgent
PaperAgent
Sep 19, 2026 · Artificial Intelligence

Looped Transformers: Recurrent Depth, Adaptive Recursion, Reasoning vs Memory

This article dissects the looped transformer architecture behind GPT-6 Astra, explaining recurrent depth, adaptive recursion mechanisms from Universal Transformer to Mixture-of-Recursions, and research showing recurrence improves multi-step reasoning without increasing knowledge capacity, while reducing training compute by 6.8–18% at scale.

Adaptive ComputationGPT-6 AstraLooped Transformer
0 likes · 12 min read
Looped Transformers: Recurrent Depth, Adaptive Recursion, Reasoning vs Memory
Machine Heart
Machine Heart
Sep 8, 2026 · Artificial Intelligence

SMELT: Looped Transformers Outperform Baselines Under Fair Budget Matching

The SMELT framework from Tsinghua and ByteDance Seed fairly compares Looped Transformers against baselines by matching compute, parameters, and KV cache, finding that looping the middle 50% of layers twice with a larger depth-width ratio consistently reduces validation loss across scales, saves 6.8–18% training compute, and yields downstream gains beyond loss reduction.

Budget MatchingLLM ArchitectureLooped Transformer
0 likes · 12 min read
SMELT: Looped Transformers Outperform Baselines Under Fair Budget Matching
Machine Heart
Machine Heart
Sep 3, 2026 · Artificial Intelligence

OpenAI Astra's Recurrent Depth Achieves 100% Exploit Success, Alarms Safety Experts

OpenAI's upcoming Astra model reportedly uses recurrent depth architecture to achieve 100% success on cybersecurity benchmarks and discover zero-day vulnerabilities, but safety experts warn that increased internal computation may undermine chain-of-thought monitoring and enable hidden planning.

AI safetyAstraLooped Transformer
0 likes · 18 min read
OpenAI Astra's Recurrent Depth Achieves 100% Exploit Success, Alarms Safety Experts