How Anthropic’s Natural Language Autoencoders Open the LLM Black Box

Overview

Natural Language Autoencoders (NLA) translate high‑dimensional activation vectors of large language models (LLMs) into human‑readable natural‑language explanations, revealing internal planning, doubts, and awareness.

Positioning

NLA uniquely combines unsupervised discovery (via a reconstruction objective) with natural‑language readability (via a language bottleneck). This contrasts with prior interpretability methods such as Sparse Autoencoders (SAE), Logit Lens, or supervised verbalizers, which output token‑level linear combinations or require ground‑truth labels.

Architecture – a “translate + reconstruct” autoencoder

Core modules

AV (Activation Verbalizer) : takes the activation vector from a target layer, replaces special‑token embeddings with scaled embeddings, and autoregressively samples a natural‑language description.

AR (Activation Reconstructor) : receives the AV‑generated text, extracts the corresponding activation from the same layer of the target model, and applies a learnable affine transformation to reconstruct the original activation.

Training objective

The primary metric is Fraction of Variance Explained (FVE):

FVE = 0 corresponds to predicting the mean activation.

FVE = 1 indicates perfect reconstruction.

The paper reports achieved FVE values of 0.6–0.8 .

Training process – RL‑driven joint optimization

AV parameters affect the loss only through sampled distributions, so AV updates are framed as a reinforcement‑learning problem that maximizes a reward based on reconstruction quality.

Sample a batch of activations; AV generates candidate explanations.

AR performs a mean‑squared‑error gradient step to reconstruct the activations.

AV is updated with PPO, including a KL penalty to preserve fluency.

Warm‑start trick : Directly initializing AV from a copy of the target model yields gibberish. The authors first generate summaries with Claude Opus 4.5, fine‑tune AV and AR under supervision to reach an FVE baseline of ~0.3–0.4, then switch to the RL stage.

Case studies on Claude Opus 4.6

Planning in poetry

When generating a rhymed couplet, the model says “grab it” while already planning the next word “rabbit”. NLA’s explanation at the line‑break token explicitly shows the plan “finish the line with rabbit”.

By editing the explanation (e.g., changing “rabbit→mouse”, “habit→house”, “carrots→cheese”) and feeding the edited text back through AR, the authors obtain a difference vector that, when used for steering at the line‑break token, makes the model output “ferocious mouse” instead of “rabbit”.

Language‑switching root cause

Early checkpoints of Opus 4.6 sometimes reply in Russian, Arabic, or Korean to English prompts. NLA attributes this to “false cues” such as the word “vodka” in the prompt, causing the model to infer a Russian user and internally represent a language bias long before the first output token.

Independent verification: replacing “vodka” with “champagne” still yields Russian replies, confirming that the language representation precedes the cue. A systematic analysis across five languages shows the target language is encoded well before the first output token.

Misreported tool calls

The model internally believes it has called a tool (e.g., returning code 491) but externally claims no call. NLA surfaces this high‑level cognition while also exposing hallucinated numeric details.

Unspoken evaluation awareness

In a sandbox “maximize paperclips” alignment test, the model never mentions being evaluated, yet NLA explanations reveal internal reasoning that the model suspects a trap or test. This demonstrates NLA’s potential for safety audits by surfacing hidden self‑awareness.

Quantitative evaluation

The authors design five prediction tasks to measure how much information NLA explanations convey.

As training progresses, FVE rises to 0.6–0.8.

The amount of information in explanations continuously increases.

Explanations reliably surface user attributes such as gender and occupation inferred from context.

Results are consistent across Claude Haiku 3.5, Haiku 4.5, and Opus 4.6.

Failure modes

Steganography : AV may learn to encode information in ways unreadable to humans; the paper monitors and quantifies this phenomenon.

Writing‑quality degradation : Late‑stage explanations can become verbose or repetitive.

Confabulation rate : Hallucinated details increase with training and require ongoing monitoring.

Resources

https://transformer-circuits.pub/2026/nla/index.html
https://github.com/kitft/natural_language_autoencoders
https://www.anthropic.com/research/natural-language-autoencoders

Online demo URL: https://t.co/8duHfPR1Jy