This article details Youku's end‑side solution for real‑time danmaku‑through‑human rendering, covering its background, value, client‑side advantages, technical challenges, OPR architecture, processing pipeline, engineering optimizations, test results, and future outlook for mobile intelligent rendering.
The article presents a comprehensive AI solution for video danmu (bullet‑screen) occlusion prevention that leverages human portrait semantic segmentation, describes dataset construction with full‑ and semi‑supervised labeling, details the encoder‑decoder model with context attention, outlines post‑processing for spatial and temporal stability, and explains deployment on cloud and edge using YKit, KwaiNN and TensorRT.
The paper presents HDES‑Net, a lightweight real‑time monocular human depth estimation and segmentation network designed for embedded platforms, using MobileNetV1 backbone with ASPP and depth‑wise separable convolutions, achieving high accuracy on CAD‑60 and EPFL‑RGBD datasets while running at up to 199.93 FPS on a Tesla P40 and 17.23 FPS on a Jetson Nano after TensorRT optimization.
LiteKit, Baidu’s mobile AI deployment framework built on Paddle Lite, delivers out‑of‑the‑box video super‑resolution, human segmentation and gesture‑recognition SDKs that reduce integration complexity to three simple steps across Objective‑C, Java and C++, achieving real‑time performance (25 FPS) while lowering development effort and platform barriers.
