Douyu’s Scalable P2P Live Streaming: Architecture, Key Tech & Future Plans

This article, based on Zhou Sha’s talk at LiveVideoStackCon 2020, shares Douyin’s self‑developed P2P live‑streaming experience, covering background, architecture, key technologies, and future plans.

1. Background of Douyin P2P

Douyin, a NASDAQ‑listed company, spends over 150 million CNY per quarter on bandwidth. Before 2019 it used commercial P2P solutions, which required high service fees. In February 2019 Douyin launched an in‑house P2P project, completed gray‑scale testing by August, and began large‑scale deployment after the National Day holiday, now handling 70‑80 % of live‑stream traffic.

2. Douyin P2P Architecture

The architecture consists of a server side and a client SDK.

Server side includes three components:

Shard cluster (P2P source): pulls full FLV streams from commercial sources, slices them, stores slices in memory for CDN back‑origin, and provides a 302 scheduling service for CDN node selection. It also maintains long‑lived TCP connections with CDN L2 nodes to push slices.

P2P scheduling system: manages resources and user access.

Tracker cluster: tracks users in the same stream, assists P2P connections and signaling.

When a user enters a live room, the client calls the get_link HTTP API, receives the stream URL and P2P configuration, contacts the scheduling system, receives a Tracker IP, establishes a WebSocket connection, and receives a list of peers. The client then downloads part of the data from CDN and the rest from peers.

2.1 P2P Scheduling System

The scheduling system contains a configuration center (operational platform) for room‑level P2P settings, a channel management service that dynamically allocates trackers and slice clusters based on viewer count, and a monitoring service that removes faulty instances and improves disaster recovery.

3. Key Technologies

3.1 Sub‑stream Mode

In sub‑stream mode the sliced FLV stream is divided into multiple logical sub‑streams. For example, with three sub‑streams, slice numbers 1,4,7 belong to sub‑stream 1; 2,5,8 to sub‑stream 2; 3,6,9 to sub‑stream 3. A user receives some slices from CDN and the remaining slices from peers belonging to other sub‑streams, achieving a theoretical sharing ratio of 66.7 %.

An “emergency window” of 4 seconds forces the client to fetch missing slices from CDN if peers do not deliver in time, ensuring smooth playback.

When a user leaves or its network degrades, the tracker reassigns new peers to maintain connectivity.

Increasing the number of sub‑streams raises the theoretical sharing ratio (e.g., six sub‑streams give ~75 % sharing), but also adds connection overhead; Douyin currently uses six sub‑streams.

3.2 WebRTC

Douyin adopted WebRTC to enable data sharing across H5, PC, Android, and iOS clients, replacing earlier Flash‑based or native UDP protocols. To reduce binary size and crash rate, the original ~1 M‑line WebRTC library was trimmed to a 30‑40 k‑line “DyRTC” core that retains only DataChannel, DTLS, and P2P components.

3.3 Data Slicing

Live streams are delivered as HTTP‑FLV. The system creates a PackageTableFile index of slice IDs and recent I‑frame positions. Slices are generated when the DTS gap exceeds one second. Each slice consists of a header and multiple 1 KB‑scale chunks, the smallest P2P distribution unit.

Two slicing approaches exist: an independent slicing service or embedding slice IDs in SEI frames during CDN transcoding. The former is CDN‑agnostic, the latter ties the solution to a specific CDN.

3.4 P2P SDK

The SDK, the largest codebase in the P2P stack, implements all network protocols (HTTP/HTTPS, WebSocket, STUN, RTMP) and timers in‑house to minimize third‑party dependencies. By consolidating most logic into a single thread, lock contention is eliminated, dramatically reducing crash rates; P2P‑related crashes no longer appear in the top‑10 crash list.

3.5 P2P Strategies

Sharing strategy groups users by ISP and region to improve peer proximity.

Traversal strategy uses WebRTC’s offer/answer flow and applies NAT‑type filtering: symmetric NAT users are excluded as seeds, and port‑restricted users are not paired with symmetric NAT peers. 4G users download via P2P but upload only through CDN.

Seed scoring evaluates online duration, connection success rate, data stability, and device type to prioritize high‑quality peers.

3.6 Sharing‑Rate Optimizations

“Base‑stream” technique reduces CDN load for a subset of users, raising the effective sharing ratio to over 90 % in six‑sub‑stream mode.

Trickle ICE accelerates peer connection establishment by exchanging ICE candidates incrementally, improving sharing rates.

Region‑level grouping (3‑4 large zones) balances peer quality versus geographic proximity.

IPv6 support adds another 10 %+ of users to the P2P pool.

4. Summary & Future Plans

Douyin’s P2P solution now supports large‑scale live streams with overall sharing ratios above 75 % (over 80 % for major rooms). Future work includes exploring QUIC‑like protocols for native connections, simplifying WebRTC encryption, extending P2P to video‑on‑demand, supporting 4K/8K bitrates with adaptive strategies, and scaling the system for overseas markets with many ISPs.