How Two‑Step Distillation Exposed Claude and GPT’s Chain‑of‑Thoughts – 116‑Page Paper Reveals a Fatal API Leak

The article reports a 116‑page paper that provides concrete evidence of a serious API flaw allowing the extraction of proprietary chain‑of‑thought (CoT) reasoning from top‑tier large language models (LLMs) such as Claude, GPT and Gemini.

Eight researchers from MATS Research, the ELLIS Tübingen Institute and other labs used Opus 4.8 as a “thinking” model and Haiku 4.5 as a “decoder”. In the first step they asked Opus a question (e.g., the largest prime factor of 8139881). Opus returned a brief summary and a 36 180‑character encrypted token block that contained the full CoT. In the second step they fed the exact block to Haiku, which faithfully reproduced the entire hidden reasoning token by token.

The vulnerability stems from the API’s design: encrypted reasoning blocks are not cryptographically bound to the original session, user or model. Consequently the same block can be reused across sessions, across different users, and even across different models within the same provider’s ecosystem. This cross‑session, cross‑user, and cross‑model interoperability, originally intended for seamless model upgrades, unintentionally opens a cheap back‑door for low‑cost models to act as universal decoders. Using Haiku 4.5’s standard pricing, decoding 10 000 reasoning traces (≈12 000 tokens each) would cost only about $720.

In the first experiment, researchers extracted 16 consecutive tokens from Opus’s reasoning and asked three other models to continue the same task. The model most similar to Opus required roughly 1 × 10¹⁰ attempts to accidentally reproduce the exact segment, while the other two needed about 1 × 10¹⁴ and 1 × 10¹⁶ attempts respectively—showing a disparity of up to a factor of 10⁶.

The second experiment fed the first 1 % of Opus’s reasoning to another model before letting it answer the full question. With only a handful of tokens as a prompt, the target model’s subsequent wording, answer structure and problem‑solving rhythm shifted dramatically toward Opus’s style, raising similarity scores from 0.17 to 0.33. Extending this test to 30 questions, 29 showed the same effect, indicating that a tiny Opus seed can steer other models to mimic its reasoning.

Beyond the technical proof‑of‑concept, the researchers harvested 6 708 public agent execution traces from GitHub and Hugging Face, reconstructing 315 320 reasoning blocks. Of these, 328 traces (≈4.9 %) leaked sensitive information, including 62 API keys, 33 passwords, 24 access tokens, 7 private keys, 30 personal email addresses, 130 names and 36 postal addresses.

These findings suggest that the costly computational barriers built by AI giants to protect their models can be bypassed for a few hundred dollars, potentially eroding the competitive moat of flagship LLMs and exposing both intellectual property and user data.