Perception‑R1: A Rule‑Based RL Method that Elevates Multimodal Model Vision

Motivation

Multimodal large language models (MLLMs) such as GPT‑4o, Gemini, Qwen‑VL and LLaVA combine language understanding with image processing but still struggle with precise object localization, accurate counting, robust OCR and complex visual reasoning.

Perception‑R1 Framework

Perception‑R1 is a post‑training framework that enhances the visual perception ability of existing MLLMs (e.g., Qwen2‑VLInstruct‑2B) by applying rule‑based reinforcement learning (RL) to learn a dedicated perception policy .

Perception Policy

Extract and understand relevant visual details from the image.

Perform logical operations on the extracted information (e.g., compare positions, identify instances, read text).

Generate the required output in the correct format (e.g., bounding‑box coordinates, counts, transcribed text).

Group Relative Policy Optimization (GRPO)

GRPO, previously successful in DeepSeek‑R1, optimizes the perception policy through the following steps:

Rollout : generate multiple attempts (e.g., 8 generations) per input; randomness is controlled by the temperature parameter.

Reward Modeling : evaluate each attempt with a task‑specific reward function (e.g., Intersection‑over‑Union for bounding‑box tasks).

Relative Comparison : compute the average reward across attempts; attempts above the average receive a positive advantage, those below receive a negative advantage.

Policy Update : adjust the model to increase the probability of high‑advantage outputs and decrease low‑advantage outputs.

Repeated Optimization : repeat the cycle over a large dataset to gradually refine the perception policy.

Reward Engineering for Vision Tasks

The reward function consists of two components:

Format Reward : +1 point for correct output structure, –1 point for incorrect format.

Answer Reward : task‑specific correctness measured by:

Visual Localization (RefCOCO): IoU between predicted and ground‑truth boxes.

Visual Counting (PixMo‑Count): Euclidean distance between predicted points and ground‑truth points after converting counting to point detection.

OCR (PageOCR): Levenshtein (edit) distance between predicted and ground‑truth text.

Multi‑Agent Reward Matching

For tasks with multiple instances (object detection, counting), Perception‑R1 uses bipartite graph matching to align predictions with ground truth:

Represent predictions and ground truth as two point sets.

Compute a potential reward (e.g., IoU) for each pair.

Apply the Hungarian algorithm to find the matching that maximizes total reward.

This yields the strongest possible learning signal for multi‑object tasks.

Experimental Evaluation

Perception‑R1 was evaluated on standard visual perception benchmarks and compared with the original Qwen2‑VL‑2B‑Instruct and specialist models.

Visual Grounding (RefCOCO/+/g): Perception‑R1 outperformed the baseline and approached specialist performance.

OCR (PageOCR): Significant improvement in edit‑distance scores.

Counting & Object Detection (COCO2017 validation): Achieved >30 AP, surpassing YOLOv3 and Faster‑RCNN.

General Image Understanding : Consistent gains across diverse tasks.

Ablation Studies and Scalability

Comprehensive ablations examined the impact of:

Reward matching versus naïve matching.

Explicit “thinking” steps in the policy.

Supervised fine‑tuning (SFT) versus RL‑driven updates.

All variants confirmed that RL‑driven policy updates improve performance. Additional experiments demonstrated that the framework scales to larger models, providing empirical evidence for future large‑scale deployments.

Conclusion

Rule‑based RL, when tailored to visual tasks, can effectively teach large models to perceive more accurately and logically. By optimizing perception policies, Perception‑R1 pushes the performance frontier of MLLMs in object detection, counting, OCR and broader image‑understanding tasks.

Paper: https://arxiv.org/pdf/2504.07954

Code: https://github.com/linkangheng/PR1