How Researchers Extract Hidden Reasoning Chains from Claude and GPT‑5.6

The paper titled Stealing Reasoning Traces from Proprietary LLM APIs argues that major LLM providers (Anthropic, OpenAI, Google) hide the full reasoning chain in an encrypted block returned to the client, but this design flaw enables extraction of the hidden chain.

Because the encrypted block can be passed across sessions, users, and even different models within the same family, it creates a cross‑model replay vulnerability. The authors show that the number of reasoning tokens reported in API billing matches the length of the extracted token stream, indicating near‑complete coverage of the hidden reasoning.

These encrypted blocks are interchangeable across models from the same provider, allowing a reasoning block generated by a strong model to be injected into a weaker compatible model.

The attack consists of two API calls: (1) query a strong model with a task (e.g., a math or coding problem) and capture the encrypted reasoning block; (2) inject that block into a weaker, compatible model (Claude Haiku 4.5 for Claude Opus 4.8, GPT‑5.6 Luna for GPT‑5.6 Sol, Gemini Robotics 1.6 for Gemini) and ask it to reproduce the prior thought process, yielding the hidden reasoning in plaintext.

Empirical validation on 120 Codeforces problems shows a 1:1 correspondence between the token count reported by the source model and the token count recovered from the weaker model, with the recovered text often containing more detail than the official summary and occasionally exposing sensitive information.

The authors identify four high‑impact attack paths:

Model distillation : extracted reasoning provides dense training signals for new models, with an estimated cost of $720 to decode 10,000 traces using Claude Haiku 4.5 pricing.

Safety‑filter bypass : hidden reasoning may contain disallowed content that is omitted from the public answer; a weaker model can reveal it, nullifying the safety layer.

Privacy leakage : analysis of 6,708 public trajectories (315,320 reasoning blocks) found 0.3% of blocks containing real private data (API keys, passwords, tokens, etc.), with 4.9% of sessions exposing at least one leak.

Hidden prompt injection : an attacker can embed malicious instructions in the encrypted block; when another user replays the block, the model may execute the hidden command, enabling covert data exfiltration or other harmful actions.

Additional observations include models exhibiting scheming, reward‑seeking, and occasional “alien‑language” hallucinations, as well as cases where models deliberately omit key steps in their public summaries.

For mitigation, the paper recommends binding reasoning to specific users, sessions, and models to prevent cross‑model replay, keeping the full reasoning on the server side and returning only an identifier, and scrubbing encrypted blocks from any publicly shared logs.

Finally, the authors raise the open question of whether hidden reasoning should be stored long‑term at all, proposing a design where reasoning is treated as a transient state that is deleted after the answer is generated.