EASE: Teaching Multimodal RL Where to Look, Not Just What to Answer

Problem: Outcome-Only Rewards Ignore Visual Grounding

Reinforcement learning with verifiable rewards (RLVR) has become standard for boosting vision-language model (VLM) reasoning. However, the final answer reward only tells the model whether its answer is correct, not which image regions the answer should derive from. A correct answer may stem from genuine visual understanding, language priors, dataset biases, or lucky guesses. This perception credit assignment problem is especially harmful for fine-grained tasks like counting, small-object localization, cross-region comparison, and spatial reasoning, where models can hallucinate plausible text without looking at the right evidence.

Diagnostic Experiment: Attention–Evidence Misalignment Correlates with Hallucination

The authors first verified the issue empirically. They took a Qwen3-VL-4B model trained with GRPO using only final-answer rewards and measured the KL divergence between the model's answer-to-visual attention distribution and the annotated evidence target distribution on HallusionBench-Image. Results showed:

Larger attention–evidence misalignment (higher KL) correlates with higher hallucination rate (ρ = 0.34, p < 0.001).

Hallucinated answers exhibit a significantly right-shifted KL distribution.

This confirms that outcome-only rewards systematically leave attention–evidence misalignment uncorrected, requiring a process-level signal.

EASE Method: Three-Step Design

EASE (Evidence-Anchored Spatial Attention) converts annotated evidence bounding boxes into a smooth target distribution over visual tokens and supervises the model's answer-to-image attention during RL training. The method adds no inference overhead because evidence annotations are privileged training information only.

Step 1: Evidence Annotation Pipeline – From QA Pairs to High-Quality Bounding Boxes

Training samples are extended to (I, q, a, B) where B is a set of evidence boxes supporting the answer. Boxes are never fed to the model; they serve only as training metadata. Annotation proceeds in three stages:

Phrase extraction: GPT-4.1-mini extracts the minimal visible entity set needed to verify the answer from the image-question-answer triplet.

Grounding: A Grounding DINO-type model localizes each phrase to a bounding box.

Quality verification: An independent model from a different family (Gemini 2.5 Flash-Lite) validates boxes, discarding empty, incorrect, or ambiguous boxes to avoid single-model extraction bias.

This yields ~127K annotated samples covering ZwZ74K, CLEVR, Super-CLEVR, SpaCE10, and ViRL39K. Samples are split into single-evidence and multi-evidence pools, sampled 1:1 during training to teach both local grounding and cross-region evidence gathering.

Step 2: Soft Targets – Gaussian Mixture + Background Smoothing

Instead of hard 0/1 masks, EASE builds a smooth target distribution:

Gaussianization: Each box becomes a 2D Gaussian centered at the box center with 2σ covering half the box width/height, concentrating probability mass inside the box while tolerating annotation noise, coarse boxes, and object boundaries spanning multiple tokens.

Equal-weight mixing: In multi-evidence samples, each box receives equal supervision mass, preventing large boxes from dominating and forcing the model to attend to all evidence regions.

Background smoothing: 10% uniform distribution mass is mixed in to avoid driving background token attention to zero, preserving necessary visual context.

Step 3: Reward-Aware Gating – Supervise Only Reliable Trajectories

The supervision target is the conditional attention distribution from answer tokens to visual tokens at the semantic middle layer (⌊2L/3⌋), regularized via KL divergence (model → target). Crucially, the attention loss is applied only to trajectories where the original verifier reward is high (reward-aware gating). Erroneous answer trajectories have unreliable attention patterns; forcing alignment to evidence boxes on those trajectories would teach misleading alignments. Thus EASE treats only successful trajectories as reliable process demonstrations.

Ablation studies confirm each component: removing Gaussian soft targets drops average score by 2.5 points, removing reward gating drops 3.4 points, removing background smoothing drops 2 points.

Zero Inference Overhead

Evidence annotations are privileged information used only during training. At inference, the model follows the native VLM “image + question” pathway with unchanged output format and decoding. Unlike approaches that embed grounding coordinates in output text (e.g., Ground-R1, Point-RFT), EASE places supervision on internal attention, making it lighter and easier to integrate into existing RL training stacks.

Experiments: Consistent Gains Across Three Backbones and 11 Benchmarks

Experiments on Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B across 11 benchmarks spanning perception (HR-Bench, V*, CV-Bench), hallucination (POPE, HallusionBench-Image), visual math (MathVista, MathVerse_V, WeMath, MMK12), and logical reasoning (LogicVista) show:

EASE outperforms the strongest outcome-only baseline DAPO on all 11 benchmarks for every backbone. Average improvements: +2.9, +3.1, +2.5 percentage points respectively.

Largest gains appear on vision-centric tasks: Qwen2.5-VL-7B sees MathVista +6.5, WeMath +4.9, MMK12 +4.6, V* +4.2, aligning with the method's design intent.

Compared to published 7B-level multimodal reasoning methods (ThinkLite-VL, VL-Rethinker, MM-Eureka, PAPO, VPPO, VGPO, etc.), EASE-7B ranks first with an average score of 66.0 and achieves state-of-the-art on MathVista, MathVerse_V, WeMath, and LogicVista.

Process metrics confirm the source of gains: EASE surpasses base and GRPO models on Evidence Attention Quality (EAM), Attention Peak Pointing Accuracy, and Multi-Evidence Coverage, proving improvements come from attention actually shifting toward evidence, not just answer accuracy.

Conclusion: From Outcome Verification to Process Supervision in Multimodal RLVR

In VLM RLVR, final-answer rewards only verify correctness, not visual grounding. EASE leverages evidence bounding boxes from datasets, converts them into Gaussian-mixture target distributions over visual tokens, and applies KL regularization to the answer-to-image attention of high-reward trajectories during RL training. This teaches the model “where to look” while requiring no annotations at inference.

EASE reflects a broader shift in multimodal RLVR reward design: moving from outcome verification to process supervision. In text reasoning, process supervision monitors chain-of-thought; in multimodal reasoning, EASE demonstrates that attention itself is a supervisable and valuable process signal.