One‑Model‑For‑All: Inception‑Level AI Try‑On/Off with Arbitrary Poses and No Masks

Summary Overview

This article reviews the research paper “One Model For All: Partial Diffusion for Unified Try‑On and Try‑Off in Any Pose” (arXiv:2508.04559) and explains its contributions, methodology, and experimental validation.

Problem Statement

Dependence on exhibition garments and segmentation masks : Existing virtual fitting methods require clean garment images and masks, limiting real‑world applicability.

Pose flexibility : Generated try‑on results are usually constrained by the pose of the reference image, preventing user‑defined poses.

Separate try‑on and try‑off tasks : Prior work treats try‑on and try‑off as independent problems, lacking a unified solution.

Scarcity of high‑quality 3D data : 3D‑based approaches suffer from insufficient high‑resolution 3D datasets, reducing realism.

Proposed Solution

Unified framework (OMFA) : Introduces OMFA, a diffusion‑model framework that simultaneously supports try‑on and try‑off without requiring exhibition garments or segmentation masks.

Partial Diffusion mechanism : Applies noise selectively to different components (e.g., garment, body, face) of the joint input, enabling fine‑grained sub‑task control and reducing redundant computation.

Bidirectional garment‑person modeling : Handles garment‑to‑person and person‑to‑garment transformations within a single architecture.

Pose freedom via SMPL‑X conditioning : Incorporates SMPL‑X pose parameters, allowing arbitrary pose and multi‑view synthesis from a single portrait image.

Technical Foundations

Diffusion Model : Uses latent‑space diffusion (LDM) to generate high‑quality images efficiently.

Partial Diffusion : Defines a binary mask indicating which components receive diffusion; only those parts are noised and denoised during training and inference.

SMPL‑X structural conditioning : Employs the parametric SMPL‑X human model to provide explicit 3D pose and shape information, enabling pose‑controlled synthesis without additional template images.

No‑mask design : The framework operates without segmentation masks, requiring only a single portrait and target pose.

Method Details

The joint input consists of the person image, garment image, and facial region. These are concatenated along the channel dimension and encoded into latent space by a VAE. During diffusion, a partial‑noise latent is constructed according to the component mask, and the UNet predicts noise only for the selected parts. The denoised latent is then split spatially, and the VAE decoder reconstructs the final garment‑person image.

SMPL‑X parameters are regressed from the input image using the 4D‑Humans pipeline, then rendered into RGB pose maps that serve as structural conditioning for the diffusion process.

Experiments

Datasets : VITON‑HD (13,679 front‑view models, 11,647 train / 2,032 test) and DeepFashion‑MultiModal (≈40,000 train / 1,100 test). Semantic masks are obtained via SCHP.

Implementation : Initialized from Stable Diffusion XL weights, fine‑tuned with AdamW. Trained on 4 NVIDIA A800 GPUs at 768×1024 resolution for 65,000 steps, batch size 8, learning rate (unspecified). Classifier‑free guidance with 0.05 random dropout of conditional features; inference uses DDIM with 50 steps and guidance scale 2.0.

Baselines : Compared against seven state‑of‑the‑art try‑on methods (LADIVTON, StableGarment, StableVTON, OOTDiffusion, IDM‑VTON, CatVTON, MV‑VTON) under a realistic setting where garment templates are unavailable, and against IDM‑VTON for multi‑pose try‑on, as well as TryoffDiff and TryoffAnyone for try‑off.

Evaluation metrics : Paired metrics – SSIM, LPIPS, FID, KID; unpaired metrics – FID, KID, CLIP‑I, DINO similarity, GPT‑4o‑mini 0‑10 quality score; DISTS for garment generation quality.

Results

Virtual Try‑On : OMFA matches or exceeds baselines on overall metrics; excels in non‑paired settings, especially CLIP‑I and DINO similarity, demonstrating superior semantic consistency.

Multi‑Pose Try‑On : Outperforms all baselines across every metric, showing robust handling of pose and view changes.

Virtual Try‑Off : Achieves the best scores on all five evaluated metrics, preserving fine‑grained texture and structural details better than TryoffDiff and TryoffAnyone.

Ablation Study : Replacing the dual‑UNet + ReferenceNet pipeline with a single UNet using partial diffusion improves performance and reduces computation. The “One Model For All” configuration yields clearer texture recovery and more complete garment outlines.

Conclusion

OMFA is a diffusion‑based unified framework for virtual try‑on and try‑off that eliminates the reliance on garment templates, segmentation masks, and fixed poses. Its novel partial diffusion mechanism enables efficient, fine‑grained control of garment‑person transformations, while SMPL‑X conditioning provides arbitrary pose synthesis from a single image. Extensive experiments on VITON‑HD and DeepFashion‑MultiModal confirm its state‑of‑the‑art performance and practical applicability.

References

[1] One Model For All: Partial Diffusion for Unified Try‑On and Try‑Off in Any Pose.