LRT: Implicit Reasoning Chains Boost Speed and Accuracy by Removing Redundant Steps

Motivated by the overthinking problem of slow‑thinking models such as OpenAI o1, DeepSeek‑R1 and Qwen‑QwQ, the authors question whether the lengthy step‑by‑step reasoning chains are all necessary. They observe that these chains often contain massive redundancy, leading to high latency and computational cost.

To investigate, they conduct experiments on the DeepSeek‑R1‑Distill‑Qwen‑7B model by randomly dropping varying proportions of tokens or reasoning steps. Even when 50% of the chain is removed, accuracy drops by only about 2 percentage points, indicating (1) a large amount of redundant information in current reasoning trajectories and (2) the model’s strong ability to filter essential information from incomplete, noisy chains.

These findings inspire the design of Latent Reasoning Tuning (LRT), which replaces the explicit chain with a compact latent vector generated by a lightweight inference network. The LRT workflow consists of three stages:

Input Encoding: The question is fed to a large model to obtain hidden state representations.

Latent Reasoning: The hidden states are passed through a small inference network, producing a fixed‑length latent reasoning vector in a single forward pass.

Answer Generation: The latent vector is concatenated with the question encoding and fed back to the large model, which directly decodes the final answer.

The training adopts a two‑stage strategy. First, supervised fine‑tuning (SFT) minimizes negative log‑likelihood to teach the inference network to generate useful latent vectors. Second, a reinforcement‑learning phase (GRPO) uses answer correctness as a reward, encouraging the network to explore better reasoning paths in latent space.

Experimental results on DeepSeek‑R1‑Distill‑Qwen‑1.5B and Qwen‑3 series models show that LRT consistently outperforms existing efficient inference methods across token budgets. For example, with a 512‑token budget, LRT improves average accuracy by 2.66% over NoThinking and by up to 5.90% over RL‑based baselines. On Qwen‑3‑4B, LRT achieves a pass@4 accuracy of 71.60%, surpassing the native non‑thinking mode by 5.82 points and delivering gains of ~7% on GSM8K and >14% on LSAT.

LRT also enables a hybrid reasoning paradigm: simple queries are answered quickly via implicit reasoning, while difficult problems can switch back to explicit slow‑thinking for deeper analysis. This modular design requires no changes to the base model parameters.

Ablation studies reveal that increasing the number of latent tokens from 64 to 256 steadily improves performance (42.53% → 48.42%). Adding the RL stage yields average gains of ~9% on in‑domain tasks and ~4.3% on out‑of‑domain tasks, confirming the importance of reinforcement learning for latent reasoning optimization. Larger base models (e.g., Qwen‑3‑8B) benefit further from more latent tokens, indicating a correlation between latent capacity and model size.

In summary, LRT demonstrates that (1) reasoning trajectories contain high redundancy and full step‑by‑step chains are not required for correct inference, (2) compressing these chains into latent vectors drastically reduces inference cost while improving accuracy, and (3) the modular, plug‑and‑play architecture supports efficient hybrid reasoning, outperforming current state‑of‑the‑art methods on a wide range of benchmarks.