Low‑Latency Live Streaming System: Challenges, Architecture, and Solutions

Due to the pandemic, many primary and secondary schools delayed opening, prompting Youku to launch a "Study at Home" plan with live courses. To ensure smooth interactive experiences during live streaming, Youku's streaming media team explored low‑latency technologies, achieving <300 ms latency between anchors and <600 ms latency between anchors and viewers.

1. Industry User‑Experience Analysis

Interactive live streaming involves three elements: main‑anchor, co‑anchor, and viewers. Typical latency requirements are:

Main‑anchor ↔ co‑anchor: audio/video call latency <300 ms.

Anchor ↔ viewer interactions (text, images, gifts, etc.) often exceed 3000 ms, leading to poor experiences such as delayed greetings, inability to coordinate game actions, or vote‑pulling during PK.

The goal is to reduce these latencies while maintaining user experience.

2. Traditional Solutions

Conventional live streaming uses real‑time communication for anchor‑to‑anchor (<300 ms) but relies on RTMP transmission, CDN distribution, and player buffering for anchor‑to‑viewer, resulting in ~3000 ms latency.

Key issues: RTMP transport, CDN hop, and player cache. CDN is largely uncontrollable, limiting latency improvements.

3. Low‑Latency Live Streaming System

The solution replaces the traditional pipeline with a fully controllable media transmission system.

3.1 Project Challenges

Integrating CDN‑style massive distribution with real‑time communication is complex. Reducing latency often conflicts with increasing buffer size for smoother playback, creating a trade‑off between latency and smoothness.

3.2 Solution Overview

The low‑latency system combines CDN, a private real‑time communication protocol, WebRTC, and cloud‑native technologies. It controls the entire transmission path, reducing end‑to‑end latency to under 415 ms while keeping smoothness high.

Only two controllable latency sources remain: the transport layer and the player buffer. The RDN system optimizes both, achieving transmission latency <118 ms and player‑side latency <415 ms. Two strategies—smoothness‑first and latency‑first—support both anchor‑to‑anchor and anchor‑to‑viewer interaction scenarios.

3.3 CDN Evolved into RDN System

The RDN system extends CDN architecture with media‑server nodes. When an anchor streams to an edge node, viewers are directed via GSLB to the nearest edge, and the stream is delivered through a custom SDK.

3.4 Transmission Delay Minimization

By optimizing transport protocols and eliminating unnecessary hops, transmission delay is reduced to the minimum possible.

3.5 Low‑Latency Player

The player incorporates a customized neteq to handle jitter and packet loss, and uses a filtered AV sync algorithm to maintain quality even under rapid speed changes.

3.6 Full‑Link Monitoring (Bianque)

The Bianque system collects logs from both client and server sides, providing end‑to‑end service quality metrics and facilitating online issue diagnosis.

4. Data Report

Compared with traditional solutions, interaction latency was reduced by 86% while maintaining smoothness and start‑up rates. The system has been stable since 2017, supporting various interactive live‑streaming scenarios such as PK competitions.

5. Reflections

The low‑latency live streaming system integrates traditional live‑streaming tech, real‑time communication, and WebRTC. It enables real‑time text, audio, and video interaction without noticeable delay, transforming the user experience for live events.

Key technologies involved:

Traditional live streaming: RTMP, CDN, ijkplayer

Real‑time communication: MCU, SIP, RTP/RTCP

WebRTC: JSEP, P2P, SFU, ICE, SDP, neteq