From Post‑hoc to Intrinsic: Cutting‑Edge Advances in Making Large Language Models More Transparent

Large language models have become increasingly powerful, yet a persistent question remains: can we truly understand why they answer or reason in a particular way, and why they sometimes fail or behave unpredictably?

Historically, most research has focused on post‑hoc interpretability —training a high‑performing but opaque model first, then applying feature attribution, probing, LogitLens, sparse auto‑encoders, causal interventions, and other external analyses. While valuable, this approach suffers from a "fidelity gap" because many explanations approximate rather than reflect the model's actual computation.

Recently, researchers have shifted toward intrinsic interpretability , embedding explainability directly into model structure, training objectives, and information‑flow pathways. In this view, explanation is no longer an add‑on but a built‑in component that influences model outputs.

The authors' survey paper, Towards Intrinsic Interpretability of Large Language Models: A Survey of Design Principles and Architectures , was accepted at ACL 2026 Main Conference. It asks whether we can transform a black box into a "glass box" system and organizes existing work into five core design paradigms.

1. Functional Transparency : Emphasizes clear, semantically meaningful computation rather than dense, entangled transformations. Representative methods include Generalized Additive Models (GAM), Neural Additive Models (NAM), Self‑Explaining Neural Networks (SENN), and Kernelized Additive Networks (KAN). The trade‑off is reduced expressive power and training efficiency.

2. Concept Alignment : Maps intermediate model variables to human‑understandable concepts (e.g., attributes, symptoms, topics). Concept Bottleneck Models (CBM) predict concepts first, then perform downstream tasks, allowing direct inspection of concept‑level errors but incurring an "alignment tax" that may limit expressive freedom.

3. Representational Decomposability : Seeks to disentangle hidden representations into independent subspaces or discrete codebooks. Examples include Backpack Language Models, which separate lexical meaning from contextual weighting, and CoCoMix, which explicitly injects higher‑level semantic concepts into generation, aiming to reduce semantic entanglement.

4. Explicit Modularization : Integrates modular structures such as Mixture‑of‑Experts (MoE) into LLMs. While classic MoE focuses on capacity and efficiency, recent work adds interpretability by simplifying expert networks, enforcing sparsity, or giving routing decisions semantic structure, making it possible to see which expert contributed to a prediction.

5. Latent Sparsity Induction : Applies sparse constraints, gating mechanisms, or structured regularization during training to encourage the model to develop clear activation pathways. Techniques like GLU/SwiGLU gates and sparse‑training regimes force selective parameter activation, revealing more interpretable computational sub‑circuits.

These paradigms are not mutually exclusive; many methods combine multiple principles, reflecting a broader design philosophy rather than isolated technical boxes.

The survey also traces the evolution of intrinsic interpretability (Figure 4), showing a shift from early low‑capacity, hand‑crafted models (e.g., GAM) to modern flexible, high‑capacity architectures that balance performance with transparency.

Key challenges identified include: (1) lack of unified definitions and evaluation metrics for "intrinsic interpretability"; (2) trade‑offs between interpretability and performance, especially at large scales; and (3) uncertain scalability of many methods beyond controlled or small‑model settings.

Overall, the field is moving from merely observing models to deliberately designing them to be understandable, auditable, and controllable, offering a systematic foundation for future research on trustworthy LLMs.