Why Accuracy Isn’t Enough: Mastering MCC for Imbalanced Classification

In machine learning, especially for classification tasks, performance evaluation is essential for selecting models, optimizing algorithms, and guiding research directions.

Common Classification Model Evaluation Methods

Accuracy

Accuracy is the most intuitive metric, representing the proportion of correctly classified samples to total samples. However, on imbalanced data it can be misleading.

Precision

Precision focuses on the correctness of positive predictions and is calculated as TP / (TP + FP).

Recall (Sensitivity)

Recall measures the proportion of actual positives that are correctly identified: TP / (TP + FN).

F1 Score

F1 score is the harmonic mean of precision and recall, providing a balanced measurement:

F1 = 2 × (Precision × Recall) / (Precision + Recall).

These metrics can be misleading on imbalanced datasets. For example, a classifier that always predicts “healthy” for a rare disease may achieve high accuracy but offers no practical value. To address this, a more comprehensive metric—Matthews Correlation Coefficient (MCC)—is needed.

Matthews Correlation Coefficient (MCC)

MCC is a performance metric for binary classification that considers TP, TN, FP, and FN, yielding a value between –1 and 1, where 1 indicates perfect prediction, 0 random prediction, and –1 total disagreement.

The calculation formula is:

(TP × TN – FP × FN) / √((TP + FP)(TP + FN)(TN + FP)(TN + FN)).

MCC is especially important for imbalanced data such as medical diagnosis or credit‑card fraud detection, where traditional metrics may be misleading. It also serves as a unified standard for comparing multiple models because it incorporates all four confusion‑matrix elements.

Python Implementation

Define the function manually:

<code>def compute_mcc(TP, TN, FP, FN):
    denominator = (TP + FP) * (TP + FN) * (TN + FP) * (TN + FN)
    if denominator == 0:
        return 0
    else:
        return (TP * TN - FP * FN) / (denominator ** 0.5)

# Example usage
TP, TN, FP, FN = 50, 40, 10, 5
mcc = compute_mcc(TP, TN, FP, FN)
print(mcc)
</code>

Or use scikit‑learn:

<code>from sklearn.metrics import matthews_corrcoef

# True and predicted labels
y_true = [1, 0, 1, 1, 0, 1, 0]
y_pred = [1, 0, 1, 0, 0, 1, 0]

mcc = matthews_corrcoef(y_true, y_pred)
print(mcc)
</code>

Evaluating classification models requires selecting metrics appropriate to the data and task; MCC provides a robust tool, especially for imbalanced datasets, and choosing the right metric is as crucial as choosing the right algorithm.