Web Live Streaming Essentials: Protocols, Formats, and Mini‑Program Guide

Web Live Streaming Essentials

Live streaming basics

A typical live streaming pipeline: capture → encoding → network transmission (push → server processing → CDN distribution) → decoding → playback.

IPB: a common video compression scheme using I‑frames (key), P‑frames (forward difference) and B‑frames (bidirectional difference).

GOP (Group of Pictures): longer GOP (larger I‑frame interval) increases B‑frame proportion, improving compression but raising CPU load during decoding.

Key quality indicators: content latency, stutter (smoothness), and first‑frame delay.

Main challenges: network conditions, multi‑person interaction, primary/secondary streams, browser compatibility, CDN support, etc.

MSE (Media Source Extensions) is a W3C standard API that enables JavaScript to construct media streams for

<video>

and

<audio>

elements, overcoming HTML5’s lack of native streaming support (e.g., FLV). It can be checked with

MediaSource.isTypeSupported()

, though iOS Safari does not support it.

File formats / containers

File/Container formats : formats that encapsulate video streams, such as FLV, AVI, MPG, VOB, MOV, MP4, etc. Codec determines how the video is encoded. MP4 can be split into fragmented MP4 (fMP4) for segment‑based playback without loading the whole file.

Codec : a digital signal encoder/decoder that compresses and decompresses audio/video. Hardware codecs can offload processing from the CPU.

Common video codecs : MPEG, H.264, RealVideo, WMV, QuickTime, etc.

Common audio codecs : PCM, WAV, OGG, APE, AAC, MP3, Vorbis, Opus.

Comparison of existing solutions

RTMP protocol

Based on TCP.

Adobe‑dominated, widely supported in China.

Browser playback relies on Flash.

Typical latency of 2–5 seconds.

RTP protocol

Real‑time Transport Protocol, standardized by IETF in 1996.

Based on UDP.

Strong real‑time performance.

Used for video surveillance, video conferencing, IP telephony.

Not supported by most CDNs or browsers.

HLS protocol

HTTP Live Streaming, an Apple‑proposed HTTP‑based streaming protocol.

HTML5 native

<video>

support; suitable for apps, but PC browsers only Safari and Edge support it.

Requires H.264 + AAC encoding.

Segmented media leads to higher latency.

flv.js

Open‑source by Bilibili; parses FLV data and repackages it into fMP4 via MSE for the

<video>

tag.

Encodes as H.264 + AAC.

Uses HTTP streaming (fetch/stream) or WebSocket for media delivery.

Latency of 2–5 seconds, first‑frame faster than RTMP.

WebRTC protocol

1. Promoted by Google and now a W3C standard.

2. Supported by modern browsers, including X5 in WeChat and QQ.

3. Based on UDP, offering low latency and strong resilience on weak networks, outperforming flv.js.

Performance comparison (CPU usage, frame rate, bitrate, latency, first‑frame):

flv.js – CPU 0.4, latency 1.5 s, first‑frame 2 s; WebRTC – CPU 1.9, latency 0.7 s, first‑frame 1.5 s.

4. Supports upstream (push) capabilities on the web.

5. Uses H.264 + OPUS encoding.

6. Provides NAT traversal via ICE.

In large‑scale scenarios, pure P2P is impractical; Tencent Classroom adopts a P2S model for massive audiences while allowing limited upstream streams.

Mini‑program + live streaming

Technical solution

Based on RTMP; the underlying transport is described as HTTP/2 but also mentions UDP, causing confusion.

The

live-pusher

and

live-player

components have no restrictions on third‑party cloud services.

Directly use Tencent Cloud video live capabilities by configuring push and playback URLs.

Push stream URL:

Play URL:

The following mini‑program demonstrates a simple audio‑video live setup that works smoothly on a local network:

Live‑pusher component (push stream):

<view id="video-box">

<live-pusher id="pusher" mode="RTC" url="{{pusher.push_url}}" autopush="true" bindstatechange="onPush"></live-pusher>

</view>

Live‑player component (play stream):

<view id="video-box">

<live-player wx:for="{{player}}" id="player_{{index}}" mode="RTC" object-fit="fillCrop" src="{{item.playUrl}}" autoplay="true" bindstatechange="onPlay"></live-player>

</view>

Can it be used together with WebRTC?

For Tencent Classroom, teachers push streams via RTMP, while WebRTC is currently limited to PC pull‑stream. On mobile, a mini‑program could let users watch live classes, but challenges include complex permission control, multi‑stream handling, and signaling for primary/secondary streams.

Reference articles

HTTP protocol introduction: http://www.ruanyifeng.com/blog/2016/08/http.html

Using flv.js for live streaming: https://github.com/gwuhaolin/blog/issues/3

Future‑oriented live streaming technology – WebRTC: https://ke.qq.com/course/226007#term_id=100266839

Mini‑program audio‑video capabilities – technical interpretation: https://segmentfault.com/a/1190000012636142

Mini‑program development tutorial: https://mp.weixin.qq.com/debug/wxadoc/dev/index.html