AI-Driven Fusion of Peking Opera Characters with Ink-Wash Painting Style Using PaddleGAN

Author Li Yilin, a second‑year student, aims to combine AIGC technology with traditional Chinese culture.

The project fuses Peking Opera characters with the ink‑wash painting style, creating videos that showcase a unique visual and artistic effect, while also presenting a public survey on cultural awareness.

Technical architecture : based on PaddleHub for image style transfer and PaddleGAN’s First Order Motion (FOM) model for facial motion transfer.

Image style transfer steps:

Python
# Install paddlehub and its model
!pip install paddlehub
# Download model
%%capture
!hub install stylepro_artistic==1.0.1

import paddlehub as hub
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt

# Set image paths
picture = '/home/aistudio/work/image_source.jpg'
style_image = '/home/aistudio/work/style.png'

# Load model
stylepro_artistic = hub.Module(name="stylepro_artistic")

# Load and convert images
content_image = Image.open(picture).convert('RGB')
style_image = Image.open(style_image).convert('RGB')
content_image = np.array(content_image)
style_image = np.array(style_image)

# Prepare input for style transfer
images = [{
    'content': content_image,
    'styles': [style_image],
    'weights': [0.3]
}]

# Perform style transfer
result = stylepro_artistic.style_transfer(images=images, visualization=True)

# Save results
!mv ./transfer_result/* ./transfer_result.jpg
!rm -rf ./transfer_result
!mv ./transfer_result.jpg ./output/transfer_result.jpg

# Visualize
fig, axes = plt.subplots(1, 3, figsize=(12, 4))
image_paths = ['/home/aistudio/work/image_source.jpg',
               '/home/aistudio/work/style.png',
               './output/transfer_result.jpg']
titles = ['Original Image', 'Style Reference Image', 'Output Result']
for i, ax in enumerate(axes):
    ax.set_title(titles[i])
    ax.axis('off')
    ax.imshow(plt.imread(image_paths[i]))
print("图片保存路径为/home/aistudio/output/transfer_result.jpg")

Facial motion transfer using PaddleGAN’s FOM model:

Python
!git clone https://gitee.com/paddlepaddle/PaddleGAN
%cd PaddleGAN/
!pip install -r requirements.txt
!pip install imageio-ffmpeg
!pip install ppgan

Run the demo (replace paths with your own video and image):

Python
%cd /home/aistudio/PaddleGAN/applications/
!mkdir output
!export PYTHONPATH=$PYTHONPATH:/home/aistudio/PaddleGAN && \
python -u tools/first-order-demo.py \
    --driving_video ~/work/京剧.MP4 \
    --source_image ~/work/京剧.png \
    --relative \
    --adapt_scale
print("生成的视频路径/home/aistudio/PaddleGAN/applications/output/result.mp4")

Add background music with MoviePy:

Python
!pip install moviepy
from moviepy.editor import *

videoclip_1 = VideoFileClip("/home/aistudio/work/京剧.MP4")
videoclip_2 = VideoFileClip("./output/result.mp4")
audio_1 = videoclip_1.audio
videoclip_3 = videoclip_2.set_audio(audio_1)
videoclip_3.write_videofile("./output/京剧result.mp4", audio_codec="aac")
print("添加音频后的视频路径为/home/aistudio/PaddleGAN/applications/output/京剧result.mp4")

Wav2Lip lip‑sync capability:

Python
%cd /home/aistudio/PaddleGAN/applications/
!python tools/wav2lip.py \
    --face /home/aistudio/43.jpeg \
    --audio /home/aistudio/京剧.m4a \
    --outfile /home/aistudio/output/pp_put.mp4 \
    --face_enhancement
print("保存的路径为/home/aistudio/output/pp_put.mp4")

The final video demonstrates how generative AI can revitalize traditional Peking Opera, making the ancient art form more appealing to younger audiences.

Project links: PaddleStudio project (https://aistudio.baidu.com/projectdetail/8021427), demonstration video on Bilibili, and related resources.