D-FINE Redefines Bounding-Box Regression to Reach State-of-the-Art Real-Time Detection

Introduction

Real‑time object detection requires both speed and accuracy. Transformer‑based detectors (DETR) avoid NMS but suffer high latency; recent RT‑DETR and LW‑DETR improve this. Two challenges remain: (1) bounding‑box regression is modeled as a Dirac‑delta point, which cannot represent local uncertainty and slows convergence; (2) real‑time detectors must respect strict computation and parameter budgets, and conventional knowledge distillation (KD) is inefficient for detection.

Preliminaries

Standard DETR predicts fixed coordinates (x, y, w, h) or edge distances (c, d) under a Dirac‑delta assumption, making the loss sensitive to small coordinate changes. GFocal replaces the Dirac‑delta with a discrete probability distribution but still relies on anchor points and lacks iterative refinement.

Method

D‑FINE introduces two components:

Fine‑grained Distribution Refinement (FDR) : converts regression into iterative optimization of a probability distribution for each box edge. Each decoder layer refines the distribution, allowing step‑wise correction and higher localization precision.

Global Optimal Localization Self‑Distillation (GO‑LSD) : transfers refined localization knowledge from deeper layers to shallower ones via bidirectional self‑distillation, improving early predictions without extra training cost.

FDR uses a non‑uniform weighting function w(·) to adjust loss curvature, enabling fine adjustments when predictions are close to ground truth and larger corrections when they are far.

The Fine‑grained Localization (FGL) loss extends Distribution Focal Loss by adding an IoU‑weighted cross‑entropy term, concentrating the distribution around low‑uncertainty predictions.

GO‑LSD aggregates Hungarian‑matched predictions from all decoder layers into a unified set and applies self‑distillation. A Decoupled Distillation Focal (DDF) loss re‑weights high‑IoU but low‑confidence predictions.

Fine‑grained Distribution Refinement

At decoder layer ℓ, the initial box B⁰ = (x_c, y_c, w, h) is converted to center coordinates and edge distances. For each edge e ∈ {left, top, right, bottom}, a discrete distribution over K bins is predicted. The logits of layer ℓ are added to the logits of layer ℓ‑1 as residuals, then normalized with softmax to obtain refined probabilities p⁽ℓ⁾_e.

The weighting function is defined as

where α and β control the curvature. When the box is close to the target, β yields a small curvature for fine adjustments; when far, α creates a steep curve for larger corrections.

The FGL loss for edge e is

where q_e is the ground‑truth bin index, τ is the temperature for KL smoothing, and IoU weighting encourages concentrated distributions for high‑quality boxes.

Global Optimal Localization Self‑Distillation

After Hungarian matching for each decoder layer, all matched predictions are merged into a global set G. For each prediction i ∈ G, the teacher distribution t_i (from the deepest layer) is distilled into the student distribution s_i (from a shallower layer) using KL divergence:

KL(t_i || s_i) = Σ_k t_i(k) * log(t_i(k) / s_i(k))

The DDF loss weights each term by the number of matched (N_m) and unmatched (N_u) predictions and by classification confidence Conf:

L_DDF = (1/N_m) Σ_i w_m * KL(t_i || s_i) + (1/N_u) Σ_j w_u * KL(t_j || s_j)

where w_m = 1 and w_u = Conf.

Experiments

On COCO val2017, D‑FINE‑L (31 M params, 91 GFLOPs) achieves 54.0 % AP at 124 FPS (8.07 ms latency) and D‑FINE‑X (62 M, 202 GFLOPs) reaches 55.8 % AP at 78 FPS (12.89 ms). Pre‑training on Objects365 raises AP to 57.1 % (L) and 59.3 % (X), surpassing YOLOv10, RT‑DETR and LW‑DETR.

Integrating FDR + GO‑LSD into Deformable DETR, DAD‑DETR, DN‑DETR and DINO improves AP by 2.0‑5.3 % without extra parameters.

Ablation from the RT‑DETR‑HGNetv2‑L baseline (53.0 % AP, 32 M, 110 GFLOPs, 9.25 ms) shows:

Removing decoder projections reduces GFLOPs to 97 and latency to 8.02 ms, AP drops to 52.4 %.

Adding a target‑gating layer restores AP to 52.8 %.

Replacing CSP with GELAN raises AP to 53.5 % (hidden dimension reduced to keep efficiency).

Non‑uniform sampling per scale (S:3, M:6, L:3) yields 52.9 % AP.

Applying the RT‑DETRv2 training schedule brings AP to 53.0 %.

Finally, adding FDR and GO‑LSD reaches 54.0 % AP with 13 % lower latency and 17 % fewer GFLOPs.

Hyper‑parameter sensitivity: best AP (54.0 %) obtained with α=0.5, β=2.0, K=9 bins, temperature τ=0.07. Larger K improves AP up to 53.7 % then saturates; extreme α/β values degrade performance.

Distillation comparison: Logit Mimicking reduces AP to 52.6 %; Feature Imitation to 52.9 %; simple Localization Distillation to 53.7 %; GO‑LSD achieves 54.5 % AP with only 6 % extra training time.

Visualization

Figure 4 shows the iterative refinement: the first decoder layer predicts a coarse box (red) and a flat probability curve; the final layer outputs a refined box (green) and a peaked distribution, illustrating the effect of the weighting function.

Conclusion

D‑FINE redefines bounding‑box regression in DETR through FDR and GO‑LSD, delivering state‑of‑the‑art accuracy‑speed trade‑offs on COCO and Objects365. Limitations include a modest gap for ultra‑light models, suggesting future work on deeper decoding layers that are dropped at inference.

Code and pretrained models: https://github.com/Peterande/D-FINE