Tencent Video Cloud Mini-program Audio/Video Solution: From Concept to Implementation

The article recounts the development of Tencent Video Cloud's audio/video capabilities for WeChat mini-programs, initiated in early 2017 when the author, Rex Chang (Chang Qing), joined the Tencent Video Cloud team and recognized the impending impact of mini-programs on mobile applications.

Initial challenges included limited support in the mini-program environment, which only offered basic <video> tag playback using system players, resulting in high-latency HLS streaming unsuitable for real-time scenarios.

After a year of SDK refinement, the team seized a collaboration opportunity with the WeChat team, embarking on a two‑week deadline to deliver a producible solution for product demonstration and acceptance.

The solution was built around two core abstractions: audio/video uplink (push) and downlink (play), implemented via the <live-pusher> and <live-player> tags.

For uplink, the SDK captures camera and microphone data, applies preprocessing (denoising, beauty filters, noise suppression), encodes the streams, and transmits them via TCP for live streaming or UDP for real‑time communication.

For downlink, the SDK receives data, uses a jitter buffer (VideoJitterBuffer) to mitigate network variability, decodes with OpenGL for rendering, and plays audio through system interfaces.

To meet stringent latency requirements (≤500 ms end‑to‑end), the team introduced delay control and UDP acceleration, adapting playback speed to network conditions and replacing TCP with UDP where appropriate.

For bidirectional video calls, additional modules were added: noise suppression, echo cancellation, QoS flow control (adjusting encoder output based on upstream bandwidth), and packet loss recovery, collectively enabling an RTC (Real‑Time Chatting) mode within the same tags.

Scaling to multi‑party calls required room management and a notification system to synchronize participant states via a server‑backed room concept and IM‑style broadcasting.

The final architecture progresses from simple live‑streaming (push+play) → low‑latency uplink/downlink → bidirectional RTC with audio processing → multi‑party conferencing via room and notification services.

The article concludes with a diagram of the mini‑program audio/video technical system and notes that a demo can be found by searching “腾讯视频云” in WeChat Mini‑programs, encouraging readers to explore the original post for hands‑on trial.