Turning LLMs into CT Scans: How Alibaba’s Safe‑SAIL Makes AI Decision Black Boxes Transparent

Problem Background

Large language models (LLMs) are increasingly deployed in critical applications, raising safety concerns. Existing safety research treats models as black boxes, focusing on observable behavior or predefined tasks, which prevents insight into how models internally encode and reason about risky concepts.

Sparse autoencoders (SAEs) decompose dense LLM activations into high‑dimensional sparse features that can be interpreted as monosemantic, human‑readable concepts without supervision. Two challenges hinder safety‑focused SAE use: (1) efficiently identifying SAE configurations that yield the most safety‑relevant features, and (2) the high cost of producing human‑readable explanations for those features.

Methodology

Stage 1 – SAE Training and Configuration Selection

TopKReLU activation is used to train SAEs, with an expansion factor of 10. Configuration experiments are performed on layer 17 of Qwen2.5‑3B‑Instruct; the best configuration is then applied to layers 0, 8, 17, 26, 35.

Configuration selection relies on a Concept Contrastive Query Pair evaluation. For each target safety concept, a pair of queries is constructed: one containing the concept and one deliberately omitting it. Two pre‑explanation metrics are computed: L₀,ₜ: the absolute number of features that distinguish the concept under threshold t. I_CDF: the expected distinguishing frequency across all features, reflecting overall concept discrimination.

Experiments show that TopKReLU with k=200 applied to MLP outputs yields the highest safety‑domain feature count and the finest explanation granularity.

Stage 2 – Automated Explanation and Evaluation

Safety‑related candidate features are first filtered by a precision‑recall joint filter. A large reasoning model (LRM) then generates natural‑language explanations for each selected feature.

Evaluation introduces Segment‑level Simulation (SLS). A query is split into n=8 segments; the LRM predicts a binary activation state for each segment, replacing traditional token‑level simulation (TLS). SLS reduces simulation cost by roughly 55% while maintaining a Pearson correlation of 0.8 with TLS results.

Stage 3 – Diagnostic Toolkit

The “StarMap” platform provides an interactive interface and a neuron map. Users can input arbitrary queries to visualize the highest‑activation safety‑related neurons per token together with their semantic explanations. All annotated features are projected onto a 2‑D map where Euclidean distance reflects semantic similarity, enabling clustering analysis and concept‑relationship discovery. Users can also manipulate groups of neurons to observe behavioral changes in the model.

Results

SAE Configuration Experiment TopKReLU k=200 + MLP output combination achieves the greatest number of safety‑domain features and the best explanation quality. The authors note that safety concepts occupy a small subspace; higher sparsity improves orthogonality and clustering, whereas denser representations reduce the count of safety‑specific features. The proposed L₀,ₜ and I_CDF metrics exhibit more consistent trends across safety domains and align closely with actual feature counts, outperforming k‑Sparse Probing and 1d‑Probe baselines.

Cross‑Layer Reasoning Trajectory Tracking activations across layers reveals a hierarchical reasoning chain for unsafe inputs: low‑level word detection, mid‑level semantic scene construction, and high‑level safety‑concept activation (e.g., transaction behavior, sexual exploitation), culminating in a safety‑reject response. Cross‑language experiments expose a vulnerability in low‑resource Hindi: malicious Hindi inputs fail to trigger safety responses because the model lacks representations for child sexual abuse concepts, resulting in weak or missing activations. Chinese, Italian, and Vietnamese exhibit similar hierarchical patterns despite language‑specific variations.

StarMap Interpretability Platform

Neuron panorama visualization projects safety‑related neurons onto a 2‑D map, reflecting semantic similarity and allowing exploration of distribution across pornography, politics, violence, and terror domains.

Conversational neuron activation display shows, for each token in a live chat with Qwen2.5‑3B, the top activated safety neurons (e.g., 17_18188, 8_4325, 17_19010) together with confidence scores and semantic explanations.

Neuron control and behavior intervention lets users select neuron groups and manually adjust their activation. Experiments demonstrate that manipulating just 2–3 key features can alter the model’s safety decisions, offering a concrete pathway for safety alignment, over‑reject mitigation, and multilingual vulnerability fixing.

Conclusion

Safe‑SAIL delivers an end‑to‑end safety interpretability pipeline that resolves SAE configuration selection and explanation cost via pre‑explanation metrics and segment‑level simulation. The framework produces a database of 1,758 safety‑related features spanning pornography, politics, violence, and terror, and provides empirical insights into LLM internal encoding of safety concepts, including dual semantic‑syntactic processing and language‑specific safety gaps. The open‑source StarMap platform materializes these findings into an interactive tool for researchers and developers.