Thinking Inertia: LLMs Keep Reasoning Even When Told Not To, Tsinghua Study Shows

Background: The Illusion of "No-Thinking" Modes

Users often tell LLMs "don't reason, just give the answer." The model appears to comply, but a new study from Tsinghua University, Oxford University, and Stanford University shows the model continues to reason internally – a phenomenon they term Thinking Inertia . Just as physical inertia keeps a moving object in motion, once an LLM starts thinking it struggles to stop.

Why the Problem Was Overlooked: Three Flawed Proxies

The community has relied on three surface-level proxies to judge whether a model is "thinking":

Prompt-side control : assuming a no-think mode means no reasoning.

Format control : assuming absence of <think> tags means no reasoning.

Length control : assuming short output means no reasoning.

The study demonstrates that all three proxies completely fail – models produce explicit reasoning even when thinking is supposedly disabled.

A Three-Layer Measurement Framework

To measure "not thinking" directly from output content, the authors split each response into pre-answer text T and final answer A , then apply three increasingly strict criteria:

L1 Empty Thinking Rate (ETR) : strictest – output contains nothing but the answer.

L2 Question Relevance (QRel) : semantic similarity between T and the question, computed with Qwen3-Embedding-4B.

L3 Explicit Inference Rate (EIR) : GPT-4.5 (blind judge) classifies T into five categories; only "explicit reasoning" counts as a hit.

Crucially, accuracy is computed separately – a model can be 100% correct while reasoning throughout, or silent but wrong. The two dimensions must be examined independently.

Six Intervention Levels: From Encouragement to Structural Suppression

The study tests a spectrum of interventions (M1–M6):

M1: THINK-ON (encourage reasoning)

M2: THINK-OFF (native thinking-off baseline)

M3: Mild natural-language instruction

M4: Stronger natural-language instruction

M5: Strongest natural-language instruction

M6: Structural bypass – pre-fill "The answer is \boxed{" to force answer-only format

Core Finding: Answer Space Openness Drives Inertia

With thinking turned off (M2), the Empty Thinking Rate (ETR) collapses as answer space opens:

True/False (BoolQ, StrategyQA) : ETR 80–100% – models mostly stay silent.

Multiple Choice (MMLU, MMLU-Pro) : ETR varies widely, 0–95%.

Open-Ended (GSM8K, MATH) : ETR near 0% – models almost always produce reasoning.

Explicit Inference Rate (EIR) shows a clean ladder: Bool < MCQ < Open . The inertia is not a binary switch but a structural pull from the task's answer space.

Reverse experiment: adding "Think step by step" with thinking off instantly restores reasoning levels, but the Bool < MCQ < Open ladder remains unchanged – interventions change total reasoning volume, not its structural distribution .

Suppression Trade-off: Harder Pressure Lowers Accuracy on Open Tasks

On open-ended tasks, moving from M2 to the strongest linguistic suppression (M5):

ETR rises from ~0% to ~40% ✅

Average accuracy drops by ~15 percentage points ❌

EIR falls from 99.69% to 62.00% – 60% of responses still contain explicit reasoning

On True/False and MCQ tasks, suppression barely hurts accuracy; on open tasks, "not thinking" costs correctness .

Same Questions, Different Answer Formats: Structure Over Difficulty

Controlled experiment: same MMLU/MMLU-Pro math questions, same Qwen3-32B, thinking off, only answer format changed:

Original MCQ : Accuracy 77.8%, ETR 10.5%

Rewritten as True/False (verify candidate): Accuracy 78.9% (≈same), ETR jumps to 62.5% , EIR drops from 87.0% to 36.9%

Rewritten as Open-Ended : Accuracy falls to 52.6%, ETR 0% , EIR 90.1%

Presence of candidate answers determines whether the model can answer directly. This is not a difficulty effect but a structural one: with options, intuition suffices; without, the model must compute.

Discipline Differences Under Fixed MCQ Format

Even with identical four-choice format, STEM subjects show highest accuracy but lowest ETR (0.3%–36.4%) and highest EIR (59.7%–85.5%). Models more proficient in a domain are more compelled to show their work – high accuracy does not imply easier silence.

Paper References

Thinking Inertia: LLMs Keep Thinking When Told Not To
https://arxiv.org/abs/2610.11765
https://thinking-inertia.github.io/
GitHub: https://github.com/thinking-inertia/code