Integrating WebRTC Real‑Time Audio/Video with WeChat Mini Programs: Architecture and Implementation Details

Background – To extend the capabilities of Tencent WeChat Mini Programs with real‑time audio/video, the existing WebRTC‑based communication platform of the 58TEG department is connected to the Mini Program environment, allowing both caller and callee sides to operate on Mini Programs with minimal changes to the original WebRTC code.

WebRTC Real‑Time Audio/Video Solution – The platform uses a Room Server to manage sessions; participants register a room, exchange signaling, and transmit media via PeerConnection. Media streams follow standard RTP/RTCP protocols.

WeChat Mini Program Audio/Video Support – Mini Programs expose live‑pusher (RTMP push) and live‑player (RTMP pull) components. Minimum versions: iOS 6.5.21, Android 6.5.19, Mini Program base library 1.7.0. The Mini Program media stack is based on RTMP.

Interoperability Architecture – A WebRTC Gateway acts as a special WebRTC endpoint that exchanges signaling with the Room Server and bridges media to a Streaming Server. The Streaming Server converts RTP/RTCP streams from WebRTC to RTMP for Mini Programs and vice‑versa.

Session Establishment – Two main cases are supported: (1) WebRTC initiates a call and the Mini Program acts as callee; (2) Mini Program initiates a call and the WebRTC side acts as callee. Both cases involve room creation, signaling via the Room Server, and ICE negotiation, with the Gateway handling Offer/Answer translation.

Media Stream Exchange – Because WebRTC uses RTP/RTCP and Mini Programs use RTMP, the Gateway forwards RTP streams to the Streaming Server, which performs protocol conversion. Each session reserves two media ports (A and B) for bidirectional forwarding.

RTP Forwarding – The Gateway receives RTP packets from WebRTC, forwards them to port A (listened by the Streaming Server), and receives converted RTP packets from port B to send back to WebRTC.

Transcoding Module – The Streaming Server converts RTP/RTCP video (H264/VP8/VP9) to RTMP‑compatible H264 and audio (opus) to AAC, enabling seamless media exchange.

Optimization of the Transcoding Path – By restricting WebRTC video to H264, the system can bypass full decoding/encoding, reducing latency. A custom H264 raw‑stream packer replaces the original codec logic, cutting call setup delay from 1‑5 s to 0.5‑1 s.

RTMP First‑Frame Optimization – To avoid black‑screen at call start, the RTMP relay buffers the first IDR frame and subsequent frames until the pull side is ready, reducing initial latency from 2‑3 s to about 1‑1.5 s.

Session Keep‑Alive – Both the Gateway and Streaming Server are monitored via heartbeats (WebSocket for the Gateway, RPC polling for the Streaming Server). Failure detection triggers session termination and removal of the faulty server from the pool; retry logic isolates failures in multi‑server deployments.

Conclusion – The presented solution successfully integrates WebRTC with WeChat Mini Programs, delivering stable, low‑latency audio/video calls while preserving the existing WebRTC infrastructure. Ongoing work will continue to refine the Gateway and Streaming Server for broader deployment.