How FireRed-Image-Edit Sets New Standards for AI-Powered Image Editing

FireRed-Image-Edit Overview

FireRed-Image-Edit is an instruction‑driven diffusion model for image editing. It jointly processes a text prompt, a latent representation of the source image, and optional reference images, enabling pixel‑level control and strong instruction comprehension.

Dataset Construction

The raw collection comprised 1.6 billion images (≈9 × 10⁸ text‑to‑image samples and ≈7 × 10⁸ edit pairs). After stratified sampling and three‑level de‑duplication—global feature clustering, signal‑to‑noise ratio filtering, and structural similarity checks—the dataset was reduced to >100 million high‑quality samples with a balanced generation/edit ratio.

Visual quality filters removed over‑exposed, under‑exposed, color‑distorted, watermarked, heavily compressed, and AI‑generated synthetic images. A synthetic data engine generated additional instruction‑controlled samples using mask‑based and skeletal‑keypoint controls, and hard‑negative examples were created for robustness. Expert double‑blind scoring refined the final annotations.

Model Architecture and System Efficiency

The backbone is a dual‑stream multimodal diffusion transformer that fuses text tokens, high‑resolution image latents, and reference‑image features. Spatial alignment across source and target images is achieved with a 3‑D unified rotation positional encoding.

Training efficiency techniques include:

Bucketed sampling that groups images with similar aspect ratios to minimise padding.

Random instruction alignment that randomly drops or shuffles reference images, forcing the model to learn semantic relationships rather than positional memorisation.

Pre‑computed visual‑language embeddings stored offline to avoid repeated encoding.

Full‑shard data parallelism with mixed‑precision (FP16) training.

High‑speed intra‑cluster networking to reduce inter‑GPU communication bottlenecks.

Progressive Training Roadmap

Training proceeds in four stages:

Base pre‑training on massive noisy web data to acquire broad world knowledge.

Continued pre‑training on higher‑resolution mixed generation/edit tasks, using progressive time‑step sampling that starts with high‑noise coarse images and gradually shifts to low‑noise fine details.

Supervised fine‑tuning on curated high‑resolution samples with strict instruction adherence. Identity‑consistency constraints preserve facial features; a dynamic weight‑decay schedule relaxes this constraint during fine‑detail synthesis.

Reinforcement learning from human feedback (RLHF) where positive‑sample reinforcement assigns higher loss weights to high‑quality outputs and a typography‑aware reward penalises misaligned or oversized text.

Exponential moving average (EMA) of model weights smooths learning curves.

Multi‑Dimensional Evaluation (REDEdit‑Bench)

REDEdit‑Bench contains 1,600+ human‑written edit pairs covering 15 scenarios (e.g., basic retouching, facial beautification, text rendering, background reconstruction, virtual try‑on). Metrics include success rate, over‑edit rate, style‑fusion score, and consistency preservation.

Blind human evaluations show FireRed‑Image‑Edit matches or exceeds top closed‑source models in instruction following, identity preservation, fine‑grained text rendering, and creative composition.

Key Technical Resources

GitHub repository: https://github.com/FireRedTeam/FireRed-Image-Edit

Hugging Face model hub: https://huggingface.co/FireRedTeam/FireRed-Image-Edit-1.0

ModelScope page: https://modelscope.cn/models/FireRedTeam/FireRed-Image-Edit-1.0