Brake Overthinking in Long‑Reasoning Models by Detecting Semantic Redundancy

Long‑thinking models such as DeepSeek‑R1 and Qwen3‑Thinking achieve high scores with extended reasoning chains, but 41–52% of generated tokens are produced after the model has already output the final answer, resulting in redundant computation.

Blind spot of existing early‑exit methods

Current inference‑time early‑exit techniques monitor the "readiness" of a trial answer by checking confidence, consistency, or repeated high‑confidence predictions. This signal only reflects answer stability and does not guarantee that the reasoning process has converged. Experiments show an average mis‑stop rate of 44% and answer‑consistency of 64%, with many premature exits where the model would later self‑correct to the correct answer.

PUMA’s semantic‑progress approach

PUMA (Progress‑aware Unified Monitoring framework for Adaptive early exit) shifts the focus to whether recent reasoning steps still generate new semantic information. A lightweight redundancy detector, built on a Qwen3‑Embedding‑0.6B model fine‑tuned with contrastive learning, flags candidate exit points where consecutive steps are highly similar (i.e., merely re‑phrasing or re‑validating existing conclusions). At these points a second stage performs answer verification: the trial answer’s confidence, consistency across candidates, and any confidence drop are examined. Only when both stages pass does PUMA halt inference; otherwise a Loop Breaker prevents endless repetition.

Experimental validation

The authors evaluated PUMA on five long‑thinking models (DeepSeek‑R1‑Distill‑Qwen 7B, 14B, 32B; Llama‑3.1‑Nemotron‑Nano‑8B; Qwen3‑30B‑A3B‑Thinking) across five challenging benchmarks (MATH‑500, AIME24, AIME25, OlympiadBench, GPQA‑Diamond). Across all settings PUMA reduced token consumption by an average of 26.2% while maintaining accuracy; in some configurations accuracy even rose because early exit avoided post‑answer drift. Real‑world speedups were observed (DeepSeek‑7B × 1.40, DeepSeek‑14B × 1.28). Compared with Answer Convergence, Dynasor, and DEER, PUMA showed a more stable accuracy‑efficiency trade‑off, and the redundancy detector added only 0.4–1.1% overhead.

Zero‑shot transferability

The semantic‑progress signal transfers without fine‑tuning to code generation (LiveCodeBench) and multimodal reasoning (MathVista/MathVision). With a similarity threshold τ_sim = 0.50, token reductions of 18–19% (code) and 23.8–33.6% (multimodal) were achieved, while pass@1 changes stayed within 1.5 points and no parameter adjustments were required.

Internalizing the signal

By treating PUMA‑selected exit points as supervision, the authors trained models with SFT, DPO, and GRPO (PUMA‑RL). Deployed without any PUMA module, PUMA‑RL attained higher average accuracy (67.0) and greater token reduction (34.9%) than the non‑trained PUMA baseline (66.2 accuracy, 24.3% token reduction).

Limitations

The approach relies on stepwise reasoning traces; when outputs are very short or difficult to segment, the semantic‑progress detector’s effectiveness diminishes.

Paper title: Stop When Reasoning Converges: Semantic‑Preserving Early Exit for Reasoning Models

ArXiv link: https://arxiv.org/abs/2605.17672

Code repository: https://github.com/giovanni-vaccarino/PUMA