How RealRestorer Bridges the Gap in Real‑World Image Restoration

Summary Overview

Problem Addressed

Poor generalization of synthetic degradation : Existing methods trained on synthetic data fail on complex real‑world degradations.

Unrealistic evaluation : Traditional PSNR/SSIM metrics require paired clean images, which are unavailable for real scenes.

Open‑source gap : Closed‑source editors show strong performance, but open‑source alternatives lag behind.

Proposed Solution

Core framework : RealRestorer fine‑tunes the open‑source Step1X‑Edit model, retaining its large‑scale DiT backbone, QwenVL text encoder, and Flux‑VAE representation, while adapting the DiT for low‑level restoration.

Key ideas : Transfer the strong priors of a large‑scale editing model to real‑world restoration using a combined synthetic‑and‑real degradation pipeline.

Technical contributions :

Constructed a degradation synthesis pipeline covering nine real‑world degradation categories with fine‑grained noise modeling, region‑wise perturbations, and web‑style effects.

Collected additional real‑world degraded images and generated high‑quality clean counterparts via high‑performance models.

Adopted a two‑stage training regime: ~1M synthetic pairs for transfer learning, followed by ~100k real pairs for supervised fine‑tuning, using a Progressively‑Mixed strategy that retains a fraction of synthetic data to avoid over‑fitting.

Applied Techniques

Large‑scale editing model transfer : Leverages the semantic prior and content modeling of editing models for complex degradations.

Synthetic + real mixed data : Simultaneously trains on both data types to balance scalability and realism.

Reference‑free benchmark : Introduces RealIR‑Bench, which evaluates restoration score (RS) via a VLM and content consistency via LPIPS, combined into a final score (FS).

Results

Open‑source SOTA : RealRestorer ranks first among open‑source methods on RealIR‑Bench and third overall, approaching top closed‑source models.

Balanced multi‑task performance: best on deblurring and low‑light enhancement, second on moiré removal, achieving 5 first‑place and 2 second‑place rankings across 9 tasks.

Stronger content consistency : Preserves structure, semantics, and fine details better than aggressive editing‑model baselines.

Zero‑shot generalization : Demonstrates capability on unseen degradations such as snowy scenes and old‑photo restoration.

Methodology

Model Design

RealRestorer fine‑tunes Step1X‑Edit; the backbone is a large DiT, the text encoder is QwenVL, and images are encoded by Flux‑VAE. During training, VAE and text encoder are frozen, and only the DiT is updated, shifting the model from high‑level editing to low‑level restoration.

Dataset Construction

The training set consists of two parts:

Synthetic Degradation Data : Clean images scraped from the web are degraded with a sophisticated pipeline that mimics real‑world artifacts, filtered by SAM‑2, MiDaS, VLM, and quality models.

Real‑World Degradation Data : Real degraded images are collected online, paired with high‑quality references generated by strong models, and filtered using CLIP, watermark detection, Qwen3‑VL, low‑level metrics, and manual review.

Training Scheme

Stage 1 – Transfer Training : Uses ~1 M synthetic pairs to transfer high‑level editing priors to restoration.

Stage 2 – Supervised Fine‑tuning : Introduces ~100 k real pairs; employs a Progressively‑Mixed strategy that mixes a small portion of synthetic data to retain broad generalization while focusing on real‑world fidelity.

Both stages train at 1024×1024 resolution.

Experiments

RealIR‑Bench contains 464 real‑world degraded images covering nine degradation types, curated for diversity and intensity.

Evaluation metrics include Restoration Score (RS) for degradation removal, LPIPS for content consistency, and a combined Final Score (FS).

Performance

RealRestorer consistently outperforms existing open‑source editors on RealIR‑Bench and approaches the performance of leading closed‑source systems.

Ablation Studies

Two‑stage training is essential: using only the first stage yields an FS peak of 0.122 but poor real‑world generalization. Adding the second stage with real data quickly surpasses the first‑stage peak, though excessive training on real data leads to over‑fitting, mitigated by early stopping.

Training solely on synthetic data results in incomplete degradation removal; training solely on real data causes over‑fitting, manifesting as object deformation, misplaced subjects, or excessive enhancement. The mixed two‑stage approach balances removal ability and structural stability.

Progressively‑Mixed Strategy

Retaining a small fraction of synthetic pairs during the second stage prevents over‑fitting to the limited real distribution and yields measurable gains in both visual stability and quantitative scores.

User Study

Thirty‑two participants ranked outputs of five top models (3200 results). Rankings aligned with automatic metrics: Nano Banana Pro (32.02%), GPT‑Image‑1.5 (23.83%), RealRestorer (21.54%). Correlation analysis (Kendall’s τ, Spearman, Pearson) shows moderate agreement between automated scores and human perception, validating RealIR‑Bench’s relevance.

Conclusion

RealRestorer fills a long‑standing gap in the open‑source community by delivering a unified, real‑world‑focused restoration solution with strong quality and content fidelity, accompanied by a dedicated benchmark. Limitations include high inference cost (28 denoising steps) and occasional failure on extreme degradations such as mirror selfies or severe physical inconsistencies.

References

[1] RealRestorer: Towards Generalizable Real‑World Image Restoration with Large‑Scale Image Editing Models