TraceForge: Legal Synthetic Reasoning Traces Beat Closed-Source Distillation

Background: The Closing Moat of Chain-of-Thought

Over the past two years, the most intense competition in AI has centered on access to the full reasoning traces (chain-of-thought) of the strongest closed-source models. Anthropic publicly accused Chinese model companies of distilling its outputs and claimed to have blocked 90%–99% of such distillation attempts. DeepSeek's V4.1-Flash-1017 release faced similar suspicions of distilling from GPT-6-Astra. Consequently, several Chinese model labs have stopped releasing new flagship models, with insiders suspecting they no longer know how to train the next generation without access to those hidden reasoning traces.

The industry's open-source catch-up has largely relied on seeing the strongest models' reasoning processes. Closed-source teams are now hiding those traces, returning only answers or high-level summaries, citing safety and intellectual property. As the article notes, "A teacher that only gives answers cannot teach a student how to think."

These teams themselves are built on distilling human knowledge — yet they forbid others from doing the same to them. — Technical report abstract, with the note "This is a bit ironic."

Previous Workarounds and Their Failure

Researchers previously showed that encrypted reasoning blocks returned by providers are interchangeable across models; replaying them could restore the hidden chain-of-thought. However, Anthropic's October 12, 2026 statement confirmed defenses against such replay, specifically naming "moving reasoning blocks between models" as a blocked technique. They report a 90%–99% reduction in distilled data volume. A frontline model-team member stated: "Distilling Claude is basically impossible now."

Thus the core question becomes: Is there a fully legal path to obtain usable reasoning traces?

TraceForge: Family-Native Trace Generation

Yan Wang's technical report "TraceForge: Distilling with Your Own Thinking" proposes a method called family-native trace . Instead of trying to recover the teacher's reasoning, the student model learns to "think like its own family":

Train a generator on 10,000 real dialogues from a same-family model (Qwen3.8-Max), masking the original chain-of-thought and asking the model to rewrite it given context and answer.

Train a discriminator (parameter count ≥ generator) to distinguish real traces from generated ones.

Run five rounds of online adversarial training : samples that fool the discriminator are used to train the discriminator; those that fail are used for preference optimization.

Critical point: No token from Claude or OpenAI ever enters the gradient computation. All training data comes from same-family models that permit legal distillation. The teacher only provides an answer; the student writes the entire reasoning process.

Experimental Results

Using Qwen3.5-9B-Base as the student, 5,000 questions, and identical training recipes, each condition trained on 5,000 samples. Three striking conclusions emerged:

It beats the "original" distillation. Synthetic traces outperformed direct distillation of Fable 5's real traces by 3.7 points on HumanEval+ and 7.1 points on MBPP+ , with the MBPP+ lead holding at every checkpoint.

It crushes the "strongest same-family" baseline. Using Qwen3.8-Max's own traces and answers yielded scores 21 and 35 points lower . A strong teacher is necessary but not sufficient — the teacher must speak in the student's language.

It leaves peer synthesis methods far behind. With GPT-6.1-SOL answers, TraceForge achieved 59.15/60.05 vs. 3.66/6.61 for a prior synthesis method — gaps of 55 and 53 points .

On authenticity: a third-party judge rated real traces at 66% probability of being genuine, adversarially trained generated traces at 68%, and after discriminator-based selection at 82%. The synthetic traces not only pass detection — they appear more genuine than the real ones.

Three Secrets Hidden in the Learning Curves

The opening curve chart reveals three phenomena:

Original distillation is highly unstable. Fable 5 traces gave HumanEval+ 66.46 at step 600, crashed to 33.54 at step 1,200, then recovered to 64.63 at step 1,600. The author reran twice to rule out scoring errors. Explanation: the teacher's trace style diverges too far from the student's family distribution; pushing loss down forces parameters far from their original region, causing instability.

The worst start yields the most stable convergence. The TraceForge + GPT-6.1-SOL condition began as the worst (0.61 / 10.32) but was the only one to improve at every checkpoint, eventually surpassing the original distillation by step 1,600.

Former high-flyers collapse due to dead loops and meaningless repetition. Both the same-family strong-model baseline and a prior synthesis method showed early peaks (48.78 at step 300; 21.95/42.59 at step 900) but plateaued. Inspection revealed increasing dead loops and redundant re-computation that worsened with training.

An Even Cleaner Legal Variant

For teams still concerned about using closed-source answers, the report offers a fully closed-source-free variant:

First, the generator writes a native chain-of-thought. Pass it to the strongest same-family model that allows distillation, let it write the answer. Then combine the trace and this answer as training data.

Result: zero closed-source tokens in any gradient. Legal risk, source disputes, and distillation fingerprints all vanish. The author reports this variant matches the main experiment's performance.

Implication: The Assumption That Hiding CoT Locks Capability Is False

If an answer alone suffices to reconstruct a usable reasoning path — and that path can be written in the student's own language even better than the original — then hiding the chain-of-thought protects not safety but merely a little time.

Technical Report Reference

Title: TraceForge: Distilling with Your Own Thinking

Author: Yan Wang (Independent Researcher)

Base Model: Qwen3.5-9B-Base; generator and discriminator are Qwen-family models

API: ALLE AI Lab ( https://alleai.cloud/cot/), currently Qwen-family only

Disclaimer: Industry events and team backgrounds cited from public reports and rumors; used only as background. Technical conclusions per the original report; each data point is a single-run result.