How CCVTON Achieves SOTA Virtual Try-On Without Large-Scale Paired Data

Background

In large e‑commerce platforms, clothing is often shown as flat images, lacking realistic model‑on‑body visuals, which hurts user purchase decisions. Virtual try‑on (VTON) aims to transfer a target garment onto a model image with high fidelity. Recent diffusion‑based VTON methods achieve impressive realism but rely heavily on large, high‑quality paired garment‑model data, which is expensive to collect and limited in diversity. Moreover, balancing garment consistency with human‑body consistency is challenging because overly small masks leak original garment details, while overly large masks destroy body parts such as hands and feet.

Method

2.1 Unified Diffusion (UDiT)

CCVTON proposes a unified diffusion Transformer (UDiT) that shares parameters for both try‑on and try‑off tasks. Input images are formed by horizontally concatenating a garment image and a model image; a binary mask switches the mode: during try‑on the garment region of the model is masked and the target garment is synthesized, while during try‑off the garment image is masked and the model image is used to extract the garment. UDiT is first pretrained on the paired VITON‑HD and DressCode datasets to acquire basic try‑on/try‑off capabilities.

2.2 Multi‑Criterion Filtering Operation (MCFO)

To obtain high‑quality pseudo‑paired data for cycle‑consistent learning, the pretrained model performs try‑off on unlabelled single‑model images. The extracted garments are then filtered by two criteria:

ViT similarity : a conditional Vision Transformer (CondViT) computes cosine similarity between the extracted garment and the garment region in the original model image, discarding samples with artifacts or large deviations.

VLM judgment : the visual‑language model Qwen‑VL semantically inspects the extracted garment for watermarks, residual body parts, or abnormal shapes.

Only samples passing both filters are kept, yielding about 700 k high‑quality pseudo‑paired images (≈70 % of the raw data).

2.3 Cycle‑Consistent Learning (CCL)

CCL is the core training strategy. For each sample, the model first extracts a garment via the try‑off branch, then feeds the extracted garment and the same model image (with a garment mask) into the try‑on branch to reconstruct the original model image, forming a "try‑off → try‑on" reconstruction loop. The original model image serves as a natural supervision signal, eliminating the need for explicit paired annotations. A perceptual regularization loss based on VGG features aligns the extracted garment with a reference garment (obtained from the MCFO pipeline) at the semantic level, avoiding over‑reliance on noisy pixel‑level supervision.

2.4 Garment‑Aware Mask Generation (GAMG)

CCVTON adopts a two‑stage mask generation strategy:

Coarse stage : an expanded cloth‑bbox mask covers a large area, providing ample space for diverse garment transformations while DensePose supplies pose priors to preserve body structure. The coarse output aligns garments correctly but may miss fine texture details.

Fine stage : based on the coarse result, the system parses garment and background regions, refines the mask using DensePose semantic cues, and produces the final high‑fidelity try‑on image. The coarse stage uses 10 denoising steps; the fine stage adds 25 steps, increasing inference time by only ~16 % compared with a single 30‑step process.

Experiments

3.1 Quantitative Results

On VITON‑HD and DressCode, CCVTON outperforms all existing methods on every metric (FID, KID, SSIM, LPIPS) in both paired and unpaired settings, establishing a new state‑of‑the‑art.

3.2 Qualitative Results

Visual comparisons show that CCVTON preserves garment texture fidelity, maintains human‑body and background consistency, and produces more realistic outputs than prior works. For example, CCVTON accurately reproduces T‑shirt patterns that other methods blur or miss, and it achieves precise geometric alignment on striped dresses.

3.3 Real‑World Scenario Results

CCVTON demonstrates strong generalization across diverse real‑world cases, including cross‑category swaps (e.g., short‑sleeve to long‑dress), complex poses, and model‑to‑model virtual try‑on.

Conclusion

CCVTON introduces a diffusion‑based, cycle‑consistent virtual try‑on framework that eliminates the paired‑data bottleneck by leveraging massive unpaired single‑model images and a two‑stage garment‑aware mask. The method achieves SOTA performance on public benchmarks and shows robust generalization in challenging real‑world scenarios.