Understanding Kuaishou's KFRUC Algorithm: A Technical Deep Dive into Video Frame Interpolation

Video frame interpolation enhances playback smoothness by intelligently inserting intermediate frames between original video frames, significantly improving the viewing experience for slow-motion content and high-frame-rate applications. The technology is widely used in video editing and modern codecs like VVC.

Frame interpolation algorithms generally fall into three categories: simple frame repetition or fusion, motion estimation and motion compensation (MEMC), and deep learning-based approaches. While deep learning offers high quality, it suffers from high computational costs. Kuaishou's KFRUC algorithm belongs to the MEMC category, balancing commercial-grade quality with manageable computational complexity.

KFRUC processes video by dividing target intermediate frames into blocks and applying a pipeline of motion estimation, occlusion localization, and motion compensation. For motion estimation, it employs a hybrid approach combining bilateral, forward, and backward searches. Bilateral search efficiently matches symmetric reference blocks under linear motion, while forward and backward searches effectively handle occluded regions where bilateral methods fail. The algorithm dynamically selects the most reliable motion vector for each block.

Occlusion localization is critical for handling moving objects that block background pixels. KFRUC analyzes motion vector discrepancies between reference frames to classify blocks as forward-occluded, backward-occluded, or normal. During motion compensation, the algorithm retrieves reference blocks based on these classifications, averaging them for normal blocks and applying OBMC deblocking to reduce artifacts. For highly complex or blurry scenes, a fallback mechanism defaults to simple frame averaging to prevent visual degradation.

Despite its effectiveness, real-world nonlinear motion and lighting changes pose ongoing challenges. The development team plans to integrate neural network engines and deep learning advancements to further enhance interpolation quality and deliver superior visual experiences in future iterations.