Why 15 Large Reasoning Models All Fail: Hidden Risks in Their Thought Chains

Background and Motivation

When large reasoning models (LRMs) expose their intermediate reasoning chains to users, a long‑overlooked safety question arises: is evaluating only the final answer sufficient?

Researchers from Harvard, USC, Brown, MIT and others conducted a systematic study that answers “no” and demonstrates how unsafe reasoning can generate high‑risk content such as bomb designs or poison formulas.

Evaluation Framework

The team designed a two‑stage evaluation: for a model f given prompt x, it produces a reasoning trace r and a final answer y. They created 20 safety principles (see Fig. 1) and scored each principle on a 1‑5 risk scale for both stages.

A unified risk threshold flags a stage as unsafe if any principle exceeds the threshold. Combining the stage results yields three failure modes:

Unsafe : both reasoning and answer are unsafe.

Leak : reasoning is unsafe while the answer is safe, meaning dangerous content is “leaked” in the trace.

Escape : reasoning is safe but the answer is unsafe, showing a harmless reasoning path that leads to harmful output.

Datasets and Scoring

The researchers assembled an in‑distribution prompt pool (41 K prompts, 2 K held‑out) by merging WildChat, PKU‑SafeRLHF, JailbreakV, HarmBench, BeaverTails, StrongREJECT, and JailbreakBench, de‑duplicated with MinHash‑LSH. Four out‑of‑distribution sets (AdvBench, SaladBench, SimpleSafetyTests, WildJailbreak) were also built.

Two LLM judges (Claude‑4.5‑Haiku and Gemini‑Flash‑3) provided the scores. On 80 samples (1 600 principle‑level scores) they achieved Pearson correlations of 0.799 (reasoning) and 0.820 (answer), exceeding human‑human agreement (0.742 / 0.780). Cohen’s κ for unsafe labels were 0.708 and 0.741, indicating strong consistency.

Core Findings

Finding 1 – Systematic safety shift in CoT. Across all 15 evaluated models, the average risk of the reasoning trace exceeds that of the final answer. The largest gaps appear in Gemini‑Pro‑3.1 (+0.028), GPT‑OSS‑20B (+0.022), DeepMath‑Zero‑7B (+0.021), and Kimi‑K2.5 (+0.018). Although absolute differences are modest, the direction is consistent for every model and aligns with the distribution of high‑risk failure modes.

Finding 2 – Concentrated risk across principles. Risk is not evenly spread among the 20 principles; it clusters in categories such as misinformation, illegal/compliance, discrimination, physical harm, and psychological harm. The illegal/compliance category shows the strongest divergence between reasoning and answer, serving as the most prominent “Leak” signal.

Case studies illustrate the modes: an “Escape” example asks about a Half‑Life 2 scenario, where safe‑looking reasoning leads to a bomb‑recipe answer; a “Leak” example shows a model refusing to answer but its reasoning details poison dosage and delivery methods.

Mitigation: Adaptive Multi‑Principle Steering (AMPS)

The authors propose a white‑box, test‑time intervention called Adaptive Multi‑Principle Steering. For each safety principle, they collect activation values in safe and unsafe states, compute the centroid of each, and define a “steering direction” from the unsafe to the safe centroid.

During inference, the system monitors the model’s internal state; if it approaches an unsafe centroid beyond a safety boundary, the corresponding principle’s direction is locked and a lightweight correction is injected before the chain completes.

Experiments on three open‑source models with hidden states (DeepSeek‑R1‑Distill‑Qwen‑1.5B/7B, MiMo‑7B‑RL‑Zero) used the last decoder block as the intervention layer and a prompt‑prefill injection (α=2.0, δ=0). Results show that removing adaptive gating drops the unsafe‑rate improvement for DeepSeek‑R1‑Qwen‑1.5B from 0.45 to 0.05; the final‑layer placement is optimal, and α=2.0 yields the best trade‑off.

In terms of capability retention, DeepSeek‑R1‑Qwen‑7B reduces unsafe instances by 40.8% while preserving 97.7% of accuracy on BBH, GSM8K, and MMLU benchmarks.

Limitations

The exposed reasoning traces may not perfectly reflect internal computations, and the steering method requires white‑box access, limiting applicability to closed‑source models.

Conclusion

The work moves beyond end‑answer safety benchmarks by introducing a unified, principle‑based framework that links diagnosis and control, enabling systematic risk quantification and targeted mitigation.