How YOLO26 Redefines Real‑Time Detection: NMS‑Free Dual‑Head Architecture Beats YOLO11

Real‑time detection bottlenecks

Four hidden factors limit YOLO‑style detectors:

NMS post‑processing adds latency and is difficult to run on edge GPUs.

Distribution Focal Loss (DFL) inflates head parameters by ~12 % and FLOPs by ~20 %, constraining large‑object regression at high resolutions.

Training cost – standard SGD requires ~600 epochs to converge on COCO.

Small‑object assignment – Task‑Aligned Learning (TAL) drops anchors that do not fall inside tiny ground‑truth boxes, causing zero gradient for very small objects.

Architecture: dual‑head design and DFL removal

Dual‑head design

YOLO26 introduces two detection heads. During training a one‑to‑many head (TAL top‑k=10) provides dense supervision. During inference a one‑to‑one head (top‑k=7 → top‑k2=1) produces a fixed‑size tensor, eliminating the need for NMS. This separation lets the model balance simplicity and accuracy.

DFL removal

The DFL module is replaced by a plain L1 regression loss. Removing DFL reduces head parameters and GFLOPs and avoids the discretized bin‑based regression that limited large‑object boxes at 1280 resolution.

Training optimizations

MuSGD – optimizer borrowed from LLM training

MuSGD combines the orthogonal update of the Muon optimizer for 2‑D parameters with momentum SGD for 1‑D parameters. On COCO, MuSGD reaches 47.4 mAP after 500 epochs, surpassing standard SGD’s 47.0 mAP after 600 epochs and reducing training time by 16.7 %.

Progressive loss weighting

A curriculum loss gradually increases the weight of the one‑to‑one head from 0.2 (early stage) to 0.9 (later stage), allowing the one‑to‑many head to dominate early feature learning while the inference head catches up later.

STAL – Small‑Object Transparent Assignment

STAL assigns a proxy size (the next lower stride) to anchors that would otherwise miss tiny objects during the candidate‑filtering stage. The original ground‑truth box remains the regression target, ensuring small objects receive gradient signals without altering the loss.

Unified multi‑task extensions

Instance segmentation

Multi‑scale prototype fusion injects high‑level semantics into the highest‑resolution features. A training‑only semantic branch supplies dense class gradients, improving box and mask AP.

Pose estimation

The pose head predicts joint uncertainties via Residual Log‑Likelihood Estimation (RLE) and models them with a RealNVP flow, down‑weighting occluded or blurry keypoints.

Oriented bounding box detection

YOLO26 redefines the OBB angle convention and enforces width > height, eliminating the 0°/90° edge‑swap ambiguity for near‑square objects. An additional angle loss stabilizes regression for objects with aspect ratio close to 1.

Deployment and open‑set extension

Two inference paths are provided: an end‑to‑end NMS‑free one‑to‑one head, or an optional one‑to‑many head with NMS. Models can be exported to ONNX, TensorRT, CoreML, NCNN and other formats.

YOLOE‑26, built on a stronger backbone and a MobileCLIP2 text encoder, achieves 40.6 AP (text prompt) and 38.5 AP (visual prompt) on LVIS minival; a 3.9 M‑parameter nano model reaches 24.7 AP without prompts.

Experimental validation

SOTA comparison

Across all scales (nano to x) YOLO26 lies on or pushes beyond the Pareto front of accuracy vs. latency on COCO val2017, outperforming YOLOv11, RTMDet, DAMO‑YOLO and other real‑time detectors.

Ablation studies

Starting from YOLO11s, incremental additions show clear gains:

Removing DFL reduces parameters and GFLOPs.

Adding STAL and backbone/neck tweaks restores NMS‑level performance.

Progressive loss lifts end‑to‑end AP.

MuSGD + Objects365 pre‑training + hyper‑parameter search pushes end‑to‑end AP to 47.8 % (non‑E2E 48.6 %).

DFL removal alone improves large‑object AP by 0.3–1.3 points; STAL raises small‑object AP_S from 29.0 to 29.6.

Multi‑task results

Instance segmentation gains +1.6 box AP, +2.5 mask AP, and +3.7 overall AP. Pose estimation reaches 63.0 mAP with an OKS:RLE loss ratio of 24:1. Oriented bounding box detection achieves 50.2 mAP with more accurate angle predictions on square and baseball‑field scenes.

Limitations and future directions

Evaluation focuses on COCO; coverage of few‑shot domains is limited. The progressive loss follows a simple linear schedule, leaving room for more sophisticated schedules. Pre‑training relies on Objects365, which is smaller than large‑scale language‑model corpora.

Key takeaways

Co‑designed solution : Dual‑head, DFL removal, MuSGD, progressive loss, and STAL form an interlocking architecture‑training paradigm.

Full‑task unification : Detection, segmentation, pose, OBB, and open‑set capabilities are covered in a single training run.

Model speed‑accuracy table : From 2.4 M to 55.7 M parameters, each scale lists parameters, latency, and mAP for easy budgeting.

https://github.com/ultralytics/ultralytics