Can LLMs Answer More Accurately While Writing Less? Introducing SHAPE’s Reasoning Tax

1. Problem

When training large language models (LLMs) for mathematical reasoning with reinforcement learning, the reward signal is sparse: models either answer correctly but produce excessive text, or generate long but incorrect reasoning, and it is unclear where the error occurs.

2. SHAPE Overview

The research team from Huawei’s Taylor Lab, Peking University, and Shanghai University of Finance and Economics proposes SHAPE (Stage‑aware Hierarchical Advantage via Potential Estimation). SHAPE equips the reasoning chain with a “milestone + reasoning tax” mechanism that tells the model whether each step is correct and charges a cost for verbosity. The method improves average accuracy by about 3% and cuts token usage by roughly 30%, and it has been accepted at ACL 2026.

Step A: Segment + Estimate Potential

SHAPE first splits a reasoning chain into semantic segments using token‑prediction entropy; high‑entropy positions indicate logical branching points and are preferred over hard line‑break cuts. At each segment boundary a short rollout is performed: the existing reasoning is used as a prompt, the model is asked to produce a final answer within max_tokens=16, and the answer‑correctness rate defines the “reasoning potential.” For example, 8 trial answers with 6 correct give a potential of 0.75, while only 1 correct yields a low potential.

Step B: Segment‑Level Reward – Reasoning Tax

Using Potential‑Based Reward Shaping (PBRS), SHAPE adds an extra reward at each step: potential increases receive positive feedback, decreases incur a penalty. The “reasoning tax” has two properties: its base is the current potential (low early potential → near‑zero tax, high later potential → higher tax) and its rate is proportional to segment length, so longer, redundant segments are penalized more heavily.

Step C: Token‑Level Credit Redistribution

Within each segment, SHAPE computes a Z‑score‑standardized importance weight for every token based on prediction entropy. High‑entropy decision tokens receive amplified reward signals, while low‑entropy routine tokens keep their original weight, providing fine‑grained guidance that is more stable than applying shaping directly on the global outcome.

3. Experimental Results

3.1 Main Experiments

Three base models (DeepSeek‑R1‑Distill‑Qwen‑1.5B, DeepScaleR‑1.5B, Qwen3‑4B) were evaluated on five math‑reasoning benchmarks. SHAPE consistently raised accuracy by about 3% on average; DeepScaleR‑1.5B improved by 7.0 pp on AIME 2024 (38.6 % → 45.6 %) and Qwen3‑4B improved by 6.2 pp on MinervaMATH. Token consumption dropped by an average of 30%, with a maximum reduction of 38.7 % (DeepSeek‑1.5B on MinervaMATH). Training curves show SHAPE maintains higher accuracy while steadily decreasing response length.

3.2 Ablation Studies

Removing entropy‑based segmentation (EBS) increases token usage by ~3%, confirming the benefit of semantic splits.

Removing token‑level credit redistribution (TCR) lowers accuracy by up to 2.0 pp on AIME 2025, highlighting the importance of fine‑grained signals.

Sensitivity analysis of the discount factor shows that a value around 0.85 balances token efficiency and performance; too lax (0.95) causes token bloat, while too aggressive (0.7) leads to premature truncation and “short‑but‑wrong” answers.

4. In‑Depth Analysis

Stage‑aware validation on ~410 k segment transitions reveals that gains from low‑potential starts contribute about 18 % more to final correctness than gains from high‑potential starts. After SHAPE training, the proportion of potential gains originating from low‑potential states rises from 40.6 % to 44.4%, indicating the model learns to focus on difficult stages.

SHAPE also allocates token budgets adaptively based on problem difficulty, producing a steeper and lower‑variance length‑vs‑difficulty curve than GRPO. Moreover, SHAPE eliminates the “reasoning collapse” observed in GRPO, where response lengths spike near the 32 k context limit; SHAPE’s distribution decays smoothly well before the limit, confirming the effectiveness of the reasoning tax.

5. Conclusion

SHAPE offers a unified mathematical framework—dynamic‑discount potential‑based reward shaping—that simultaneously addresses three core challenges in LLM reasoning: measuring step‑wise progress, perceiving stage difficulty, and enforcing token‑efficiency constraints. Beyond the reported accuracy and efficiency gains, the introduction of a reasoning tax provides a novel design paradigm for understanding and optimizing LLM inference.