Mastering ROC Curves: How to Plot and Compute AUC for Binary Classification

Introduction

“Without measurement, there is no science.” – Mendeleev. In computer science, especially machine learning, measuring and evaluating models is equally crucial. Selecting appropriate evaluation methods enables rapid detection of issues during model selection and training, and iterative optimization.

Model Evaluation Overview

Model evaluation consists of offline and online stages. Different tasks—classification, ranking, regression, sequence prediction—require different metrics. Understanding the precise definitions of metrics such as ROC and AUC, and choosing the right ones, is a core skill for algorithm engineers.

Problem Statement

For a binary classification problem, how do you plot the ROC curve and compute the corresponding AUC? What advantages does the ROC curve have over the PR (Precision‑Recall) curve?

Background

Binary classifiers are the most common models. Common metrics include precision, recall, F‑score, PR curve, and especially the ROC‑based AUC, which is often the primary indicator of performance.

Answer

The ROC (Receiver Operating Characteristic) curve plots the false‑positive rate (FPR) on the x‑axis and the true‑positive rate (TPR) on the y‑axis. AUC (Area Under the Curve) is the integral of this curve.

FPR = FP / N, where N is the number of negative samples; TPR = TP / P, where P is the number of positive samples.

Example: In a hospital test of 10 patients (3 positive, 7 negative), if the model correctly identifies 2 positives (TP=2) and misclassifies 1 negative as positive (FP=1), then TPR = 2/3 and FPR = 1/7, representing a point (1/7, 2/3) on the ROC curve.

The ROC curve is generated by varying the classification threshold (the “cut‑off point”). For each threshold, compute TPR and FPR, plot the point, and connect all points.

When the threshold is set to +∞, all samples are predicted negative, giving the origin point (0,0). As the threshold decreases, more samples are predicted positive, moving the point upward and rightward until the curve reaches (1,1).

Another intuitive method: start at (0,0), sort samples by predicted score, then for each positive sample move up by 1/P, and for each negative sample move right by 1/N, until (1,1) is reached.

Since the ROC curve typically lies above the diagonal y = x, the AUC ranges between 0.5 and 1. A larger AUC indicates better ranking of positive samples.

Compared with the PR curve, the ROC curve is more stable when the class distribution changes. PR curves can vary dramatically with different ratios of positive to negative samples, while ROC curves remain largely unchanged, making ROC a more reliable indicator for imbalanced datasets such as advertising conversion models.

Next Topic Preview

Upcoming: SVM model – exploring linear separability of projected points on the hyperplane.