Instrumental Variable Based Causal Inference and Generalizable Causal Learning

Research Background Deep learning, while powerful, suffers from lack of interpretability and stability because it focuses on statistical association rather than causal reasoning. Complex data contain three types of relationships: causation, confounding bias, and selection bias, motivating the need for causal inference techniques.

Introducing Causality into Machine Learning Judea Pearl’s three‑step causal model (association, intervention, counterfactual) highlights that most current ML models operate at the association level. To achieve stable and explainable decisions, causal reasoning must be incorporated.

Instrumental Variable (IV) Fundamentals An IV Z must satisfy three conditions: (1) Z is correlated with the treatment T, (2) Z affects the outcome Y only through T, and (3) Z is independent of unobserved confounders U. When valid, causal effect can be estimated via two‑stage regression: first regress T on Z (and observed confounders X) to obtain \(\hat{T}\), then regress Y on \(\hat{T}\) (and X).

Challenges with IVs In practice, valid IVs are rare; they may be weak, violate independence, or be nonlinear. High‑dimensional settings make matching difficult, and many methods (propensity score, doubly robust) assume binary treatments and treat all observed variables as confounders.

Advances and Extensions Recent work proposes (1) learning representations \(\Phi(X)\) that are independent of \(\hat{T}\) before the second‑stage regression, (2) handling invalid IVs by modeling latent confounder representations, (3) generating IVs automatically (AutoIV) by enforcing relevance to T and independence from unobserved confounders, and (4) using domain indices or social network features as surrogate IVs when explicit IVs are absent.

Applications IV methods have been applied to evaluate the causal impact of military service on income (Nobel‑winning study), to improve robustness in image classification across domains, and to address interference in social networks (NetIV) where friends’ attributes serve as IVs.

Conclusion Incorporating instrumental variables into machine learning bridges the gap between association‑based models and true causal reasoning, enhancing interpretability, stability, and generalization across heterogeneous data environments.