How MVPainter Achieves Accurate, High‑Detail 3D Texture Generation with Multi‑View Diffusion

Introduction

As generative AI advances, 3D content creation is shifting from geometric modeling toward photorealistic reconstruction, making texture quality a critical bottleneck. Existing pipelines separate geometry reconstruction and texture synthesis, and while geometry methods have progressed, high‑quality, open‑source texture generation remains limited.

Core Challenges

The authors identify three main challenges: (1) fidelity to the reference image’s style and structure, (2) precise alignment with complex geometry, and (3) rich high‑frequency detail in the generated textures.

Data Construction

A multi‑view data pipeline filters and augments public 3D datasets by removing low‑quality textures, enhancing lighting and viewpoints, and ensuring semantic validity, resulting in a diverse, high‑detail training set.

Approach

MVPainter employs a multi‑view diffusion model conditioned on normal and depth maps via a Union ControlNet architecture, enabling texture generation that respects both style and geometry. A PBR attribute extractor jointly learns BaseColor, Roughness, and Metallic maps from the generated views. Training proceeds in three stages: (1) UNet pre‑training without geometric control, (2) ControlNet fine‑tuning with frozen UNet to learn geometry‑guided synthesis, and (3) joint fine‑tuning on the curated high‑quality dataset.

Evaluation

The authors build a human‑aligned evaluation system using a visual‑language model (VLM) to assess appearance fidelity, geometric alignment, and texture richness, producing ELO scores for each method. Quantitative comparisons show MVPainter surpasses existing open‑source solutions on PSNR, LPIPS, and the VLM‑based metrics. Visual results demonstrate superior detail and accurate alignment across multiple viewpoints, including PBR‑compatible outputs.

Conclusion

MVPainter delivers a complete, reproducible, open‑source system that addresses the three core challenges of 3D texture generation, achieving state‑of‑the‑art results across diverse geometry sources and laying a foundation for future research in free‑view synthesis and real‑time inference.