Skip to content
Xiaomi's PROVE Wins ACM MM 2026 Outstanding Paper Award for Visual Object Removal Evaluation

Xiaomi's PROVE Wins ACM MM 2026 Outstanding Paper Award for Visual Object Removal Evaluation

Xiaomi's PROVE paper wins ACM MM 2026 Outstanding Paper Award by introducing a perceptual evaluation framework with RC-S and RC-T metrics plus the PROVE-Bench benchmark, enabling quantifiable, human-aligned assessment of visual object removal that improves Xiaomi's magic eraser models.

Xiaomi Tech
Xiaomi Tech
Xiaomi Tech

Research Background

As diffusion models popularize image and video editing, removing objects and naturally inpainting backgrounds has become common in content creation, post-production, and privacy handling. However, a fundamental problem persists: how to measure removal quality. Object removal is a one-to-many problem — multiple plausible backgrounds exist for the same removed object, so no single ground truth exists. Existing metrics fail collectively: full-reference metrics like PSNR, SSIM, and LPIPS favor conservative copy-paste results; no-reference metrics like ReMOVE and CFD give higher scores to blurrier outputs; temporal metrics like TC and TF computed over full frames dilute local flickering. Data-wise, real videos lack paired ground-truth without the object, while synthetic datasets with ground truth cannot reflect real-world complexity.

PROVE Framework

The team proposes PROVE, a perceptual evaluation framework consisting of two metrics and a benchmark:

RC-S (Spatial Consistency) : Splits the mask into independent objects by connected components, crops each with a 1/3 outward expansion as context, and uses a sliding window on DINOv2 semantic features to compare the feature distribution of the erased region with surrounding background via Maximum Mean Discrepancy (MMD), capturing human-visible local inconsistencies.

RC-T (Temporal Consistency) : Takes the union of masks across adjacent frames, crops uniformly, and tracks feature distribution drift window-by-window only in the intersection region where both frames are repaired, avoiding whole-frame averaging that masks local flicker.

PROVE-Bench : Includes PROVE-M with 80 real-captured paired videos (input/mask/ground-truth, 81 frames at 1080p, augmented with motion simulation for handheld, push-pull, and follow shots) and PROVE-H with 100 challenging videos (crowds, dynamic backgrounds, high texture, complex reflections, fast motion) without ground truth for stress testing.

Experimental Results

The team systematically compared 13 evaluation metrics, including traditional image quality metrics (PSNR, SSIM, LPIPS) and object-removal-specific metrics (ReMOVE, CFD). Experiments covered 6 image and video datasets, 11 models, nearly 20 validation experiments, and a subjective study with 20 participants.

Results show RC-S achieves average Kendall's τ of 0.59 and Spearman's ρ of 0.66 with human judgment, outperforming m-LPIPS (0.52/0.58), ReMOVE (0.26/0.29), and CFD (0.16/0.18), ranking first on average across 6 benchmarks. Further experiments demonstrate that as removal regions become blurrier, ReMOVE and CFD scores paradoxically improve, while RC-S consistently reflects quality degradation. For video temporal stability, RC-T effectively detects flicker and discontinuity caused by random frame drops or swaps, whereas traditional TC and TF show almost no response.

Application and Open Source

This research shifts object removal evaluation from experience-based to quantifiable and reproducible, and has been integrated into Xiaomi's product pipeline. The PROVE framework assists optimization of Xiaomi's Image Magic Eraser models deployed on Xiaomi 18 series, Xiaomi 18 Fold, Xiaomi 17 series, and REDMI K100 series, and is also used for other Xiaomi imaging model iterations and training data cleaning. PROVE's code (Apache 2.0) and data are open source, with potential extensions to short-video creation, e-commerce material cleanup, film post-production, and privacy de-identification.

Project page: https://xiaomi-research.github.io/prove/

Code: https://github.com/xiaomi-research/prove

Data: https://huggingface.co/datasets/xiaomi-research/PROVE-Bench

Paper: https://arxiv.org/abs/2605.14534

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

XiaomiACM MM 2026perceptual evaluationPROVEPROVE-BenchRC-SRC-Tvisual object removal
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.