AI-Powered High-Resolution Portrait Restoration Using StyleGAN and Face Parsing

Image clarity impacts everyday life, from viewing experience to preserving valuable memories and aiding security. Degradation can arise from blur, noise, compression, and other factors, making high‑quality restoration essential.

The project aims to mitigate quality loss in portrait images by leveraging AI to recover true facial details, overcoming limitations of capture devices and transmission.

Problem Analysis & Data Construction – Image degradation is modeled as a combination of blur, down‑sampling, noise, and compression. By analyzing common degradation types (Gaussian/shot/impulse noise, Gaussian/motion/defocus/resize blur, JPEG compression, pixelation), synthetic paired datasets are created from high‑resolution images with varied degradation levels and random combinations to mimic real‑world distributions.

Technical Solution

The overall pipeline (see ) consists of face detection and landmark localization, face alignment, face parsing to obtain semantic masks, and a high‑resolution portrait model that processes the face region. The enhanced face is then inverse‑transformed and composited back onto the original background.

Model Design

To incorporate facial structural priors, the face parsing mask is fed alongside the original image into an encoder‑decoder framework. Inspired by StyleGAN, a generative architecture (see ) modulates feature maps at each level using style maps derived from both inputs, allowing fine‑grained control from coarse to detailed synthesis while preserving identity without explicit identity loss.

Training employs L1, perceptual, and adversarial losses, with a diverse batch composition to cover multiple degradation types. Fine‑tuning reduces reliance on the parsing mask, enabling robust restoration even when the mask is imperfect.

Results & Outlook

The system delivers high‑fidelity, identity‑preserving portrait restoration, simultaneously denoising background regions. It is already deployed in the “Yi Sweet Camera” product, and future work will continue to improve algorithmic performance and expand to broader visual enhancement scenarios.

References

[1] A Style‑Based Generator Architecture for Generative Adversarial Networks, 2019 [2] Analyzing and Improving the Image Quality of StyleGAN, 2020 [3] stylegan‑encoder. https://github.com/pbaylies/stylegan-encoder, 2019 [4] Image2stylegan: How to embed images into the StyleGAN latent space, 2020 [5] Encoding in Style: a StyleGAN Encoder for Image‑to‑Image Translation, 2021 [6] Additive Angular Margin Loss for Deep Face Recognition, 2018