Self‑Explaining Natural Language Models: Collaborative Game Rationalization and Solutions for Spurious Correlations

Explainability has become increasingly critical in sectors such as finance, healthcare, and law, prompting regulatory bodies like the EU to require transparent AI systems. While large language models (e.g., GPT‑3.5, GPT‑4) can generate fluent explanations, they remain black‑box models whose reliability is questionable in high‑stakes scenarios.

To address this, the authors discuss the RNP (Recursive Neural Predictors) framework, a collaborative game‑theoretic approach that pairs an explainer (generator) with a predictor. The explainer selects a rationale sub‑set of the input, which the predictor then uses for its final prediction, guaranteeing faithfulness through a “Certification of Exclusion.”

The framework faces two major issues: spurious correlations (feature correlation) and degeneration, where the generator may produce meaningless rationales that the predictor overfits to. The authors propose three solutions:

Folded Rationalization (FR): folds the two‑stage RNP into a single shared‑parameter model, aligning the learning pace of generator and predictor and improving performance on filtered beer‑review data.

Decoupled Rationalization (DR): keeps the original architecture but reduces the predictor’s learning rate relative to the generator, effectively lowering the predictor’s Lipschitz constant and mitigating degeneration without sacrificing accuracy.

Multi‑Generator Rationalization (MGR): employs multiple generators with varied learning rates to produce diverse rationales, handling both spurious correlations and degeneration while maintaining inference efficiency comparable to standard RNP.

Experimental results on both filtered and unfiltered datasets show that FR, DR, and MGR each achieve significant improvements over baseline methods, with DR often attaining the best trade‑off between rationale quality and predictor performance.

Future work will explore causal inference for explainability, transfer of the collaborative game insights to other domains such as knowledge graphs and recommendation systems, and further theoretical analysis of Lipschitz continuity in cooperative adversarial settings.