How Explainable Boosting Machines (EBM) Combine Accuracy and Interpretability

EBM Model

Traditional statistical models are simple and easy to interpret but often suffer from low accuracy. To achieve higher predictive performance while retaining interpretability, Yin Lou introduced the Explainable Boosting Machine (EBM) in 2012. Built on the generalized additive model framework, EBM fits each feature with boosting trees and employs the FAST algorithm to discover important pairwise interaction effects, incorporating the top interactions into the model for improved accuracy. The model is available in Microsoft’s open‑source interpret package.

Model Definition

EBM (Explainable Boosting Machine) embeds boosting trees within a generalized additive model, enabling both regression and classification with accuracy comparable to complex models. Its clear structure provides strong interpretability, allowing both global and local explanations of model results.

The generalized additive model forms the backbone of EBM, and suitable fitting methods such as gradient boosting or backward fitting are used. FAST efficiently ranks all pairwise feature interactions, selecting the most influential ones for inclusion.

Identifying Second‑Order Interactions

While traditional methods would retrain the model for each interaction, FAST quickly computes the residual sum of squares (RSS) for every pair of features, sorts them by impact, and selects the top interactions. This reduces computational cost dramatically, especially when the number of feature pairs is large.

For each selected pair, a depth‑2 tree partitions the two‑dimensional feature space into four regions, capturing the interaction without exhaustive search. A greedy search finds the split points that minimize RSS, and a lookup table pre‑computes losses for all possible split combinations, limiting the number of bins to at most 256.

Model Interpretability

EBM provides both global and local explanations. Global interpretation shows each feature’s shape function and overall importance, measured by the standard deviation of its contribution. Local interpretation explains individual predictions by detailing the contribution of each feature and interaction for that sample.

Example visualizations include feature importance rankings and shape function plots with corresponding feature value distributions. A local explanation for a breast‑cancer prediction illustrates how feature contributions sum to a high probability, matching the true label.

Code Implementation

<code>from sklearn.datasets import load_iris, load_breast_cancer
import pandas as pd
from sklearn.model_selection import train_test_split
from interpret import show
from interpret.glassbox import ExplainableBoostingClassifier

data = load_breast_cancer()
df = pd.DataFrame(data['data'], columns=data['feature_names'])
df['target'] = data['target']
X, y = df.iloc[:, :-1], df['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.7)

ebm = ExplainableBoostingClassifier()
ebm.fit(X_train, y_train)
ebm_global = ebm.explain_global()
show(ebm_global)
</code>

Advantages and Disadvantages

EBM’s inclusion of second‑order interactions significantly boosts accuracy while maintaining strong interpretability. Key strengths include:

Automatic detection and ranking of pairwise interactions, with heatmaps to visualize their effects.

More accurate interaction effects than RuleFit, reducing spurious interactions via the FAST algorithm.

Fast computation thanks to pre‑computed interaction lookup tables.

Simplified data preprocessing and feature engineering, requiring only basic handling of missing or outlier values.

Reference:

Shaw Ping, Yang Jianying, Su Sida. Interpretable Machine Learning: Models, Methods, and Practices .