Understanding Video Super-Resolution: Principles, Common Defects, and Practical Enhancement Techniques

In 2014 the SRCNN paper introduced deep convolutional networks for image super‑resolution, marking the beginning of AI‑driven up‑scaling. Since then, both the quality and speed of super‑resolution have improved, and Bilibili’s own models have made the technology widely applicable to video content. However, deep‑learning‑based super‑resolution still suffers from inherent limitations that can be mitigated by targeted human intervention.

The mathematical basis of super‑resolution follows the Nyquist‑Shannon sampling theorem: video resolution acts as the sampling frequency, and higher spatial frequencies correspond to sharper details. When the original resolution is low, high‑frequency information is irrevocably lost, and no interpolation can recover it. Fourier analysis shows that down‑sampling removes high‑frequency components, which appear as a contraction of the outer region in the frequency spectrum.

Super‑resolution is not merely image stretching. While interpolation cannot increase true detail, AI models can synthesize plausible high‑frequency content, albeit sometimes introducing artifacts. Common defects observed in AI‑upscaled anime include:

Loss of stylistic noise (the “oil‑painting” effect) because models prioritize low‑frequency reconstruction.

Inconsistent line depth and sharpness, especially for wide lines that are treated as solid blocks.

Weak‑texture smearing, where subtle background details are erased.

Temporal flicker or jitter when frame‑to‑frame variations cause inconsistent enhancements.

To address these issues, traditional preprocessing can be combined with AI models. Noise‑layer separation using BM3D denoising followed by subtraction isolates the stylistic grain, which can be re‑added after super‑resolution to preserve the original look. BM3D is a block‑matching 3‑D filter that excels at Gaussian noise removal while retaining fine details.

Line‑depth inconsistencies can be mitigated by first down‑sampling the source to its original production resolution (using a Descale algorithm) and then up‑scaling with the AI model. Adjusting the target resolution until lines appear most coherent yields better results. Additional post‑processing such as Gaussian blur (sigma ≈ 1.2) can soften overly sharp edges, while unsharp masking or line‑thickening can enhance weak lines.

For weak‑texture preservation, a Canny edge detector (TCanny) is applied to separate strong edges, weak textures, and flat regions. Strong edges are taken from the AI result, weak textures are chosen from either the AI output or the original frame depending on visual quality, and flat areas can remain unchanged. This selective merging retains fine background details without the “oil‑painting” artifact.

The workflow demonstrated includes BM3D denoising, noise‑layer recombination, resolution adjustment, Gaussian blur, and TCanny‑based texture merging, resulting in a final video that maintains the original artistic style while significantly improving perceived sharpness.

Reference: Kostadin Dabov et al., “Image denoising with block‑matching and 3D filtering”, Institute of Signal Processing, Tampere University of Technology.