How Cloud + Endpoint + Service Is Redefining Audio‑Video Communication

Audio‑Video Industry Development

The audio‑video sector has progressed from the black‑and‑white era of the 1970s through digital, standard‑definition, high‑definition, and now to immersive 4K/8K, high‑frame‑rate, HDR, and cloud‑native services, becoming a primary means for cross‑regional communication, meetings, and recruitment at Alibaba.

Cloud + Endpoint + Service Model

Cloud: Platform cloudization from PaaS to SaaS, spanning private and public clouds, delivering all services via the cloud.

Endpoint: Compatibility with PSTN, VoIP, conference‑room equipment, mobile, PC, web, and Android devices.

Service: Integrated offerings such as SMS, voice, IM, audio‑video, call‑center, cloud‑customer‑service, and AI‑enhanced features.

Audio‑video is now widely used in B2B, C2C, and B2C scenarios.

Video Encoding

Two dominant codec families dominate the market: the standardized H.264/H.265 (and upcoming VVC) and the royalty‑free open‑source VP8/VP9/AV1. Alibaba participates in AOM and contributes to VVC development to avoid codec lock‑in.

Encoding for Different Scenarios

On‑demand streaming prioritizes compression efficiency, live streaming demands low latency, and real‑time communication requires the highest latency tolerance and robustness, necessitating a balance between efficiency and quality.

Bitrate‑Resolution Pairing

Fixed bitrate and resolution are insufficient for Adaptive Bitrate (ABR) streaming; dynamic pairing based on content complexity and network conditions is needed, using convex‑hull analysis to find optimal trade‑offs.

Quality Metrics

Video quality is measured subjectively by MOS and objectively by PSNR, SSIM, and VMAF; Alibaba’s metrics also incorporate stalling and network conditions.

Adaptive Encoding

Content‑Adaptive Encoding customizes compression parameters per video, scene, GOP, or frame, guided by defined quality metrics; machine learning can predict optimal settings, and ROI‑based encoding reduces bitrate while preserving perceived quality.

Audio‑Video Server Network Architecture

Real‑time audio‑video servers employ three models: Mesh, MCU (Multi‑point Control Unit), and SFU (Selective Forwarding Unit). MCU centralizes media processing but adds CPU load and latency, while SFU offers low latency and high throughput with higher client bandwidth requirements. Alibaba adopts a hybrid SFU + MCU approach, cascading servers across regions to minimize first‑ and last‑mile latency.

Network Bandwidth Evaluation

Bandwidth assessment is critical for real‑time calls; Alibaba’s algorithms enable rapid server‑side updates without client upgrades, prioritize audio under weak networks, and convey network status to users to improve experience.

Conclusion

Audio‑video services require end‑to‑end quality monitoring beyond simple QoS metrics; continuous data‑driven optimization and proactive network management are essential to deliver seamless, immersive communication experiences.