Human‑Perception‑Based End‑Cloud Super‑Resolution: Cutting Bandwidth, Boosting Quality

LiveVideoStackCon 2023 in Shanghai gathered leading audio‑video engineers to discuss the pressing challenge of delivering immersive media experiences at lower cost. The conference highlighted two technical talks from Xiaohongshu’s audio‑video team: a human‑perception‑based end‑cloud super‑resolution framework and a data‑driven approach to interpreting AB‑test results.

End‑Cloud Super‑Resolution Framework

The team introduced an innovative framework that combines cloud‑side video processing with on‑device super‑resolution, guided by human visual perception metrics. By offloading heavy computation to the cloud, the solution achieves quantifiable super‑resolution quality on the device, high integration coverage, and significant bandwidth savings. An AI‑powered no‑reference video quality metric aligns objective scores with perceived visual quality, enabling large‑scale user‑experience quantification.

Scientific Interpretation of AB Experiments

The second talk addressed the complexities of AB testing in data‑driven product development. While AB testing is essential for rapid iteration, interpreting results can be confusing when some metrics stagnate while others fluctuate. The presentation covered the statistical foundations of AB testing, the role of significance testing, and pitfalls of relying solely on p‑values. Real‑world examples from Xiaohongshu’s client‑side super‑resolution rollout demonstrated how significance aids decision‑making, yet must be complemented with broader business context.

Balancing Compute, Cost, and User Experience

Discussion emphasized the classic trade‑off triangle among compute resources, cost, and user experience. Static analysis suggests increasing compute or cost to improve quality, while dynamic optimization leverages algorithmic advances—such as newer codecs saving 30‑50% bitrate for equal quality—to simultaneously enhance experience and reduce bandwidth expenses. Strategies like time‑based bitrate adjustments aligned with CDN peak‑hour pricing were also explored.

Future Directions and Open Challenges

Looking ahead, the team outlined ongoing work on H.265 deployment, AV1 experimentation, and preparation for H.266. They highlighted the need for intelligent, end‑to‑end video processing pipelines that integrate high‑level semantic understanding with low‑level image enhancement. Key challenges include building high‑quality, diverse datasets for AI‑driven quality assessment and developing multi‑modal models that combine perception, encoding, and content semantics.

Practical Insights for Quality Evaluation

To validate visual improvements, the team employs a two‑stage evaluation: controlled expert and crowd‑sourced assessments to capture subjective quality, followed by large‑scale AB testing to confirm gains in production. They also described a no‑reference AI quality model that extracts global and local texture features, captures spatiotemporal dependencies, and leverages self‑supervised learning to compensate for limited labeled data.