Peeking Inside Claude: How Anthropic Uncovers LLM Reasoning

Introduction

Large language models (LLMs) are increasingly deployed, yet their internal reasoning remains opaque. Anthropic’s recent papers introduce interpretability tools that insert, extract, and modify “concept features” inside Claude, enabling observation and intervention of its internal computation.

Microscopic view of model thinking

By treating concept features like neuronal activity, researchers can ask “how did Claude arrive at this answer?” and can suppress or inject specific concepts to see how outputs change.

Cross‑language sharing

Claude does not contain separate language‑specific modules. Instead, a shared set of concept features constitutes a universal “thinking language”. When a user requests a synonym or translation, Claude activates core features (e.g., “small”, “opposite”) and then maps the resulting concept into the target language. Scaling the model strengthens this shared representation, suggesting strong knowledge‑transfer across languages.

Pre‑planning in rhymed poetry

Claude does not wait until the line ends to select a rhyme. After generating the first line it pre‑selects candidate rhyme words, adjusts internal features accordingly, and then generates subsequent lines to fit the chosen rhyme. Experiments show that suppressing the concept “rabbit” causes Claude to substitute “habit”, while injecting a non‑rhyming concept such as “green” forces the model to adapt its output.

Parallel mental‑arithmetic strategies

When asked to compute 36 + 59, Claude activates multiple parallel computation paths:

One path produces a rough estimate.

Another path refines digit‑level precision.

The final answer combines signals from both paths.

Claude does not expose an explicit algorithm; it can arrive at the correct result without a human‑readable step‑by‑step trace.

Multi‑step reasoning via concept chaining

In a “state‑capital” query, Claude first activates the fact “Dallas is in Texas”, then links it to “Texas capital is Austin”, and finally outputs “Austin”. Replacing the intermediate concept “Texas” with “California” changes the answer to “Sacramento”, demonstrating genuine intermediate reasoning rather than direct memorization.

Hallucination mechanism

Claude’s default response to unknown queries is refusal. If the model mistakenly activates a “known entity” feature without factual support, it suppresses the refusal circuit and generates a plausible‑looking but incorrect answer. Researchers can deliberately trigger such hallucinations by injecting false entity concepts.

Jailbreak vs. safety mechanisms

Prompt engineering can coax Claude to begin emitting disallowed content (e.g., bomb‑making instructions). Two internal drives clash: a safety module that blocks prohibited information and a language‑coherence drive that pushes the model to complete the sentence. The model may briefly output unsafe text before the safety system overrides it with a refusal.

Implications for system design

Interpretability as a safety foundation : Real‑time monitoring of internal features can detect false reasoning, jailbreak attempts, or bias.

Parallel reasoning informs fine‑tuning : Because Claude uses multiple internal strategies, adaptation may require modifying feature‑level representations rather than only prompt engineering.

Scalable self‑analysis : Current interpretability methods are costly; automating feature extraction and leveraging AI‑assisted analysis will be needed for longer reasoning chains.

References

Anthropic, “Circuit tracing: Revealing computational graphs in language models” and “On the biology of a large language model”. Available at https://www.anthropic.com/research/tracing-thoughts-language-model

Code example

相关阅读：