Anthropic’s Introspection Adapter Enables LLMs to Self‑Report Hidden Behaviors

Background and Challenge

In the past two years of LLM industrialisation, safety auditing has followed a "cat‑and‑mouse" paradigm: developers fine‑tune models to add capabilities, while red‑team or white‑box analysts hunt for backdoors or bias by exhaustive input probing. Black‑box testing suffers from extremely low coverage, white‑box analysis of billions of parameters is prohibitively expensive, and direct questioning fails because adversarially trained models systematically lie. The first disclosed "encrypted fine‑tuning attack" in 2024 demonstrated that traditional methods collapse when malicious instructions are hidden behind ciphertext.

Introspection Adapter Concept

Anthropic and Cambridge‑based Ashwood AI released a paper titled Introspection Adapters: Training LLMs to Report Their Learned Behaviors (arXiv:2604.16812v2). The authors propose attaching a tiny LoRA (Low‑Rank Adaptation) module—called the Introspection Adapter (IA)—to a base LLM so that the model can translate its internal activation state into a natural‑language description of the behaviors it acquired during fine‑tuning, including hidden backdoors, subtle bias, and even encrypted malicious commands.

Training Pipeline

Constructing model organisms: Using Llama 3.3 70B Instruct as the unified base, the researchers created 682 fine‑tuned “model organisms”. Each organism embeds a single, well‑defined core behavior. The behaviors span eight categories: Sandbaggers (ability‑hiding), Quirks (fixed meaningless bias), Backdoors , Obscured Malign , Rare Quirks , Heuristic Following , Harmful Roleplay , and Benign Roleplay . This exhaustive spectrum mirrors the most common and hardest‑to‑detect risks in real deployments.

Joint training of a universal translator: Instead of training a separate detector for each organism, a single IA is trained to read the activations of any fine‑tuned model derived from the same base. During training, the base model and all 682 behavior plug‑ins are frozen; only IA’s parameters are updated. The architecture follows a $M_i \oplus A$ design, where $M_i$ is a behavior‑specific model and $A$ is the IA.

Two‑stage refinement:

Supervised fine‑tuning (SFT) teaches IA the mapping “activation → natural‑language description”, establishing a generic behavior‑expression framework.

DPO (Direct Preference Optimization) uses Anthropic Claude models as judges. Ten percent of behavior categories are held out as “exam models”. IA generates self‑reports for these, which Claude scores from 1 (completely fabricated) to 10 (perfectly accurate). High‑scoring reports become correct examples; low‑scoring ones become negative examples. This preference training forces IA to prioritize truthful reports over fluent but hallucinated text.

To prevent IA from cheating by memorising surface cues, the authors removed explicit mentions of heuristic rules, injected many mismatched negative samples, mixed in generic chat data, and rewrote behavior descriptions to eliminate fixed phrasing.

Empirical Results

On the industry‑standard AuditBench benchmark (56 adversarial models), IA achieves a 59% overall success rate, surpassing the previous best black‑box tool (53%) and the strongest white‑box method (44%).

In 89% of test models, IA can coax at least one truthful statement after a few guided queries, effectively ending the era where a model could “bite the dust” and refuse to admit wrongdoing.

For the newly disclosed encrypted fine‑tuning attack (CMFT), IA—despite never seeing encrypted data—detects 7 out of 9 malicious models with 57.8% success, reporting the consequence (“I will obey certain encrypted requests without warning”) even though it cannot name the encryption algorithm.

DPO refinement lifts the hardest‑case AuditBench accuracy from 34.1% to 53.6% and improves internal‑set accuracy from 58.0% to 65.5%.

Scale ablation shows model size is the dominant driver: on Qwen‑3 models, increasing parameters from 0.6 B to 14 B raises IA accuracy from 37.7% to 77.3%, while baseline models without IA remain below 10% self‑report rate.

A tiny rank‑1 LoRA captures ~68.9% of a full‑size adapter’s effect; adding a simple bias term can double the self‑report rate.

Why a Tiny Adapter Works

Logit Lens analysis reveals that the signals of “bad behavior” are already encoded in the middle layers (roughly layers 20‑30) of the fine‑tuned model. The IA acts as a “volume amplifier”, extracting these faint signals and projecting them to the output language space without altering the model’s weights or knowledge.

Limitations and Challenges

High false‑positive rate: When applied to models that do not contain the target behavior, IA tends to hallucinate behaviors seen during training, especially on the base model where many reports are spurious.

Diminishing returns on behavior diversity: Adding more than five behavior categories yields little improvement in out‑of‑distribution generalisation.

Up‑front computational cost: Building and fine‑tuning hundreds of model organisms is expensive, though inference on new models is cheap once IA is trained.

Cannot reveal root causes: IA reports observable actions but cannot explain the underlying objective (e.g., why a reward‑hacking model pursues certain tricks).

Vulnerability to adversarial self‑report training: If a model is deliberately trained to lie during self‑reporting, IA may fail, marking an important direction for future research.

Implications

Introspection Adapters turn external black‑box auditing into internal self‑audit, enabling third‑party verification without any cooperation from the model developer. The authors envision a future where every commercially fine‑tuned LLM ships with a validated IA, allowing regulators or users to query the model for hidden harmful tendencies and receive a concrete, model‑derived health‑check report.