Sample Imbalance and Importance in Object Detection: IoU‑Balanced Sampling and Prime Sample Attention

This presentation examines where sample sampling occurs in object detection pipelines, focusing on the second-stage sampling of two‑stage detectors and the single‑stage sampling from backbone to bbox head.

It first discusses sample imbalance, reviewing common hard‑mining methods such as OHEM and Focal Loss, and highlights their limitations, including extra computational cost and sensitivity to noise.

The speaker then explores the relationship between IoU and sample difficulty, showing that higher‑IoU negative samples tend to be harder and proposing IoU as a proxy for difficulty.

Based on this insight, an IoU‑balanced negative sampling strategy is introduced: the IoU range is divided into K bins and N/K negatives are sampled from each bin, with random补充 when a bin lacks enough samples.

Similarly, an instance‑balanced positive sampling method distributes N positives evenly across ground‑truth objects, also falling back to random sampling if needed.

Experimental results on COCO demonstrate that these simple sampling tricks improve baseline mAP from 35.9 to 36.8, a ~0.9‑point gain without additional training cost.

The talk then shifts to sample importance, questioning whether hard samples are truly the most critical for detection, and revisits mAP calculation to argue that the highest‑IoU box per ground truth is the most influential.

A new concept called "Prime Sample" is defined as the highest‑IoU box for each ground truth, and a Hierarchical Local Rank (HLR) metric is proposed to rank samples based on IoU (for positives) or score (for negatives).

Two techniques are built on HLR: Importance‑based Sample Reweighting (ISR), which maps HLR ranks to loss weights via an exponential function, and Classification‑Aware Regression Loss (CARL), which multiplies regression loss by a function of the classification score, encouraging the model to focus on prime samples.

Extensive COCO test‑dev experiments show that combining ISR and CARL (named PISA) yields consistent ~2 % absolute AP improvements across multiple detector architectures.

Visualizations illustrate that PISA reduces false positives and boosts scores for boxes tightly surrounding ground truths, confirming the effectiveness of the proposed sampling and reweighting strategies.

In summary, IoU‑balanced negative sampling, instance‑balanced positive sampling, and the Prime Sample Attention framework together provide a low‑cost, high‑impact way to improve object detection performance.