Bilibili's Engineering Practices for HLS (fMP4) Live Streaming

In the fast‑evolving field of audio‑video live streaming, new standards and complex scenarios demand a next‑generation streaming protocol. Bilibili has pioneered the large‑scale adoption of the HLS (fMP4) protocol in China, and this article shares the engineering experience of that rollout.

Background and Motivation – Traditional live streaming relies on RTMP/HTTP‑FLV, which, despite a decade of maturity, suffers from outdated standards, limited codec support (e.g., H.265, AV1), extra transcoding overhead on browsers, and poor compatibility with international ecosystems.

Why HLS (fMP4) – The fMP4 variant of HLS is maintained by the MPEG standards body, supports modern codecs, is natively accepted by major browsers (eliminating FLV transcoding), integrates easily with CDN file delivery, offers robust DRM capabilities, and enables efficient P2P file‑based distribution.

HLS Protocol Overview – HLS (HTTP Live Streaming) segments live media into small files referenced by an M3U8 playlist. Players fetch the playlist, then download successive segments to achieve continuous playback. Two versions exist: MPEG‑TS‑based and fMP4‑based; the latter, standardized in RFC 8216 (2017), uses fragmented MP4 containers that allow finer‑grained chunking suitable for live streams.

Production Architecture Design – Bilibili’s live‑production pipeline uses a clustered slicing service that writes segments both to memory and object storage, provides hot‑standby failover, employs a tiered back‑source strategy, and leverages HTTP 302 for distributed scheduling. Memory‑mapped disks accelerate I/O for high‑throughput segment writing, while asynchronous persistence to object storage supports replay and fault‑tolerant fallback.

Back‑Source Architecture Design – A dynamic back‑source system queries CDN nodes for the optimal origin address, allowing real‑time node up/down and reducing latency. HTTP 302 redirects are also used for adaptive degradation during failures.

HLS Startup Optimization – To minimize start‑up latency, Bilibili adopts 1‑second segments and ensures the first segment begins with a key frame. The #EXT‑X‑START tag forces playback from the first segment. Additionally, CMAF/fMP4 chunking lets the client decode after receiving only the initial few frames of a segment, achieving a 2‑3× reduction in first‑frame latency compared to traditional HLS.

Live Replay Implementation – By extending the lifespan of generated segments and producing a parallel timestamped playlist (e.g., utc.m3u8), the system offers seamless replay without altering the live pipeline.

Back‑Source Pre‑Warm Optimization – HLS uses short‑lived HTTP connections, allowing rapid retries. Bilibili exploits HTTP 1.1 keep‑alive and CMAF chunked transfer to begin serving partially written segments to CDN nodes, effectively pre‑warming caches and improving back‑source hit rates for small, frequently requested fragments.

Future Outlook – Leveraging fMP4’s extensibility, Bilibili plans to integrate DRM, adaptive bitrate, digital watermarking, Dolby Atmos/VR audio, and other advanced features. The successful large‑scale deployment of HLS (fMP4) positions the platform to move beyond legacy RTMP constraints and adopt next‑generation streaming technologies.