Xiaomi's PROVE Wins ACM MM 2026 Outstanding Paper Award for Visual Object Removal Evaluation
Xiaomi's PROVE paper wins ACM MM 2026 Outstanding Paper Award by introducing a perceptual evaluation framework with RC-S and RC-T metrics plus the PROVE-Bench benchmark, enabling quantifiable, human-aligned assessment of visual object removal that improves Xiaomi's magic eraser models.
Research Background
As diffusion models popularize image and video editing, removing objects and naturally inpainting backgrounds has become common in content creation, post-production, and privacy handling. However, a fundamental problem persists: how to measure removal quality. Object removal is a one-to-many problem — multiple plausible backgrounds exist for the same removed object, so no single ground truth exists. Existing metrics fail collectively: full-reference metrics like PSNR, SSIM, and LPIPS favor conservative copy-paste results; no-reference metrics like ReMOVE and CFD give higher scores to blurrier outputs; temporal metrics like TC and TF computed over full frames dilute local flickering. Data-wise, real videos lack paired ground-truth without the object, while synthetic datasets with ground truth cannot reflect real-world complexity.
PROVE Framework
The team proposes PROVE, a perceptual evaluation framework consisting of two metrics and a benchmark:
RC-S (Spatial Consistency) : Splits the mask into independent objects by connected components, crops each with a 1/3 outward expansion as context, and uses a sliding window on DINOv2 semantic features to compare the feature distribution of the erased region with surrounding background via Maximum Mean Discrepancy (MMD), capturing human-visible local inconsistencies.
RC-T (Temporal Consistency) : Takes the union of masks across adjacent frames, crops uniformly, and tracks feature distribution drift window-by-window only in the intersection region where both frames are repaired, avoiding whole-frame averaging that masks local flicker.
PROVE-Bench : Includes PROVE-M with 80 real-captured paired videos (input/mask/ground-truth, 81 frames at 1080p, augmented with motion simulation for handheld, push-pull, and follow shots) and PROVE-H with 100 challenging videos (crowds, dynamic backgrounds, high texture, complex reflections, fast motion) without ground truth for stress testing.
Experimental Results
The team systematically compared 13 evaluation metrics, including traditional image quality metrics (PSNR, SSIM, LPIPS) and object-removal-specific metrics (ReMOVE, CFD). Experiments covered 6 image and video datasets, 11 models, nearly 20 validation experiments, and a subjective study with 20 participants.
Results show RC-S achieves average Kendall's τ of 0.59 and Spearman's ρ of 0.66 with human judgment, outperforming m-LPIPS (0.52/0.58), ReMOVE (0.26/0.29), and CFD (0.16/0.18), ranking first on average across 6 benchmarks. Further experiments demonstrate that as removal regions become blurrier, ReMOVE and CFD scores paradoxically improve, while RC-S consistently reflects quality degradation. For video temporal stability, RC-T effectively detects flicker and discontinuity caused by random frame drops or swaps, whereas traditional TC and TF show almost no response.
Application and Open Source
This research shifts object removal evaluation from experience-based to quantifiable and reproducible, and has been integrated into Xiaomi's product pipeline. The PROVE framework assists optimization of Xiaomi's Image Magic Eraser models deployed on Xiaomi 18 series, Xiaomi 18 Fold, Xiaomi 17 series, and REDMI K100 series, and is also used for other Xiaomi imaging model iterations and training data cleaning. PROVE's code (Apache 2.0) and data are open source, with potential extensions to short-video creation, e-commerce material cleanup, film post-production, and privacy de-identification.
Project page: https://xiaomi-research.github.io/prove/
Code: https://github.com/xiaomi-research/prove
Data: https://huggingface.co/datasets/xiaomi-research/PROVE-Bench
Paper: https://arxiv.org/abs/2605.14534
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
