How Kuaishou’s Y‑Tech Fixes Background Distortion in Portrait Beautification

Background

Features such as "slim face", "small head" and "long legs" have become standard in beauty apps, but when the portrait is deformed the background often warps as well, leading to user complaints like "the wall is crooked". Correcting background distortion is essential for a natural visual experience.

Algorithm Framework Overview

To address these challenges, the Kuaishou Y‑Tech beauty algorithm team designed an optimization pipeline that uses semantic information such as straight lines in the image. The framework, illustrated in Figure 2, can correct background distortion for all beautification operations on mobile devices.

Key Algorithm Details

3.1 Line Segment Detection and Distribution in Triangular Mesh

Background feature points are pre‑placed by sampling points along edges that correspond to facial contour keypoints, then each line segment is divided into four parts to generate additional points, forming a triangular mesh over the background (about 150 points in total). Figure 3 shows the mesh before and after the "small head" effect.

3.2 Slope Preservation Constraint

The algorithm assumes that the slope of a straight line in the background should remain unchanged after beautification. By detecting line segments in the original image and enforcing that the slopes before and after the "small head" operation are equal, a set of constraints is built into the optimization objective.

3.3 Triangle Shape Preservation Constraint

Some triangles contain no lines, making the slope‑only constraint unstable. To keep the solution stable, a shape‑preserving term is added that minimizes the change in edge lengths of each triangle, ensuring the triangle geometry remains consistent.

3.4 Final Optimization Objective

Combining the slope and shape constraints yields a nonlinear optimization problem solved with Gauss‑Newton or Levenberg‑Marquardt methods. Figure 7 shows the triangular mesh before (red) and after (blue) optimization, and Figure 8 presents the final corrected image compared with the original and uncorrected "small head" result.

Conclusion and Outlook

The Y‑Tech background distortion correction algorithm preserves line slopes while maintaining high‑quality beautification, runs efficiently on mobile devices, and supports all beautification operations. It has been deployed in Kuaishou’s Y‑Tech Camera and other apps, and future work will extend the technique to body‑slimming and other scenarios.

References

Line Segment Detection Papers. https://github.com/lh9171338/Line-Segment-Detection-Papers

IGARASHI, T., MOSCOVICH, T., AND HUGHES, J. F. 2005. As‑rigid‑as‑possible shape manipulation. ACM Transactions on Graphics 24, 3 (Aug.), 1134–114.

R. Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall. LSD: A fast line segment detector with a false detection control. IEEE TPAMI, 32:722–732, 2010.

A. Vakhitov and V. Lempitsky. Learnable line segment descriptor for visual SLAM. IEEE Access, 7:39923–39934, 2019.

L. Zhang and R. Koch. Efficient and robust line segment matching based on LBD descriptor. JVCIR, 24(7):794–805, 2013.

L. Nie et al. Learning edge‑preserved image stitching from large‑baseline deep homography. arXiv:2012.06194, 2020.

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014.

C.-H. Chang, Y. Sato, and Y.-Y. Chuang. Shape‑preserving half‑projective warps for image stitching. CVPR, 2014.