How the Evaluation‑Verification Reward Boosts Consistency in Multi‑Reference Image Editing

Problem

Multi‑reference image editing requires a model to understand several reference images and a textual instruction, then generate edits that preserve object identity, scene layout, visual harmony, and obey the command. Existing methods often lose reference details, introduce scene pollution, misinterpret instructions, or produce unnatural visual fusion.

EVR Framework

Multi‑dimensional assessment The editing quality is decomposed into reference consistency, scene consistency, visual harmony, instruction consistency, and overall image quality, avoiding the shortcomings of a single overall score.

Evaluation‑Verification reward paradigm For each dimension an evaluator first generates fine‑grained visual statements; a verifier then checks whether the edited image actually supports those statements. Unreliable or hallucinated judgments are filtered out, yielding a more stable and granular reward signal.

Semantic‑aligned data construction Training data are generated so that reference objects, target scenes, and edit instructions are mutually aligned, ensuring natural replacement or insertion relationships and eliminating invalid samples at the source.

Research Background

Industry demand Advertising, content creation, and similar domains frequently need to combine multiple reference objects, scenes, or styles into a single image. High‑quality multi‑reference editing can reduce production costs and increase controllability.

Core challenges The model must retain fine details of each reference, keep the original scene layout, follow textual commands, and ensure consistent lighting, scale, perspective, and semantic relations.

Limitations of existing methods Using multimodal large models directly as reward models leads to long‑chain hallucinations, while short scalar scores cannot capture complex visual judgments. An overall score mixes many evaluation dimensions, making the reward signal unstable for reinforcement‑learning fine‑tuning.

Key Contributions

Decoupling evaluation and verification EVR turns open‑ended evaluation into verification of concrete visual statements, allowing the reward model to rely on checkable evidence rather than opaque reasoning.

Multi‑dimensional reward design By modeling reference consistency, scene consistency, visual harmony, instruction consistency, and image quality separately, EVR provides fine‑grained optimization directions for the editor.

Reducing multimodal hallucination The verifier filters out evaluation evidence that lacks image support, mitigating hallucinated judgments common in large‑model evaluators.

Real‑world generalization Although training primarily uses a dual‑reference setup, the model remains stable when presented with real‑world samples or more than two references, demonstrating cross‑scene robustness.

Experimental Results

Quantitative and qualitative experiments show that EVR‑enhanced reinforcement‑learning fine‑tuning significantly improves multi‑reference editing performance.

Multi‑dimensional quality improvement Compared with a baseline model, EVR yields noticeable gains in reference consistency, visual harmony, and instruction consistency, better preserving reference details and reducing scene pollution.

Reward mechanism effectiveness Ablation that removes the verifier causes performance drops on dimensions requiring complex visual reasoning, confirming the importance of the verification step.

External evaluation and user study Combined external multimodal assessments, human‑preference evaluations, and similarity metrics indicate that EVR provides a more stable and reliable optimization signal for multi‑reference editing tasks.

Conclusion

Starting from the reward‑modeling difficulty in multi‑reference image editing, EVR proposes a “evaluate‑then‑verify” multi‑dimensional reward mechanism that alleviates hallucinations of multimodal evaluators and supplies reliable feedback for reinforcement‑learning fine‑tuning. Future work aims to extend EVR to more complex multi‑object composition, controllable image generation, and broader visual content creation tools.