Tagged articles

Looped Transformer

3 articles · Page 1 of 1

Sep 8, 2026 · Artificial Intelligence

SMELT: Looped Transformers Outperform Baselines Under Fair Budget Matching

The SMELT framework from Tsinghua and ByteDance Seed fairly compares Looped Transformers against baselines by matching compute, parameters, and KV cache, finding that looping the middle 50% of layers twice with a larger depth-width ratio consistently reduces validation loss across scales, saves 6.8–18% training compute, and yields downstream gains beyond loss reduction.

Budget MatchingLLM ArchitectureLooped Transformer

0 likes · 12 min read

SMELT: Looped Transformers Outperform Baselines Under Fair Budget Matching

Machine Heart

Sep 3, 2026 · Artificial Intelligence

OpenAI Astra's Recurrent Depth Achieves 100% Exploit Success, Alarms Safety Experts

OpenAI's upcoming Astra model reportedly uses recurrent depth architecture to achieve 100% success on cybersecurity benchmarks and discover zero-day vulnerabilities, but safety experts warn that increased internal computation may undermine chain-of-thought monitoring and enable hidden planning.

AI safetyAstraLooped Transformer

0 likes · 18 min read

OpenAI Astra's Recurrent Depth Achieves 100% Exploit Success, Alarms Safety Experts

Machine Heart

May 21, 2026 · Artificial Intelligence

Can Small Models Overthink? TaH Skips 93% Redundant Iterations and Boosts Accuracy

TaH, a selective latent‑iteration method for small language models, identifies and avoids unnecessary token‑level loops, cutting about 93% of extra iterations while delivering a stable 3.0%‑6.8% accuracy gain across nine math, QA, and code benchmarks.

LLM reasoningLooped TransformerTaH

0 likes · 14 min read

Can Small Models Overthink? TaH Skips 93% Redundant Iterations and Boosts Accuracy