WebRTC Audio‑Video Synchronization: Principles, Standards, and Implementation Details (Part 1)

This article provides a detailed technical explanation of audio‑video synchronization in WebRTC, covering the relevant international standards, the design of the synchronizer, delay adjustment algorithms, and the concrete implementation in the WebRTC code base.

Audio‑Video Synchronization Standards – Three major standards are introduced: ITU‑R BT.1359 (1998), ATSC IS/191 (2003), and EBU R37 (2007). The most widely used standard in practice is ITU‑R BT.1359‑1, which defines a five‑level rating system for synchronization quality.

Synchronizer Architecture – The synchronizer receives three inputs: the desired audio target delay, the desired video target delay, and the relative delay of the most recent audio‑video packet pair. It outputs the adjusted audio and video target delays as well as the relative delay (audio‑video target delay difference minus the recent packet pair delay).

Delay Adjustment – The target delay is computed as the maximum of the desired target delay and the minimum playback delay. A sliding‑average filter (window size k = 4) smooths the relative delay to reduce noise. The algorithm then limits the adjustment step to ±80 ms; if the absolute difference is less than 30 ms the streams are considered synchronized and no adjustment is performed.

Adjustment Strategies – When video is slower, the video delay is reduced (or audio delay increased) until the video catches up; when audio is slower, the opposite actions are taken. The final output consists of the minimal playback delays audio_delay_.extra_ms and video_delay_.extra_ms .

Audio Delay Setting – The calculated audio delay is applied via SetMinimumPlayoutDelay , which updates the DelayManager . The delay is bounded by MinimumDelayUpperBound (75 % of the max packet buffer size) and influences the effective minimum delay used for network jitter compensation.

Video Delay Setting – The video delay is stored in syncable_minimum_playout_delay_ms_ , propagated to VCMTiming , and finally used in the render pipeline. The render time is the sum of the smoothed frame time and the actual video delay.

Rendering Time Calculation – The render time is derived from the expected receive time (computed by TimestampExtrapolator using a Kalman filter) and the current jitter buffer level. The algorithm selects the maximum among jitter, decode time, render time, and the minimal playback delay to obtain the target delay for each frame.

Measurement Method (Hardware‑Based) – A practical measurement setup is described, requiring a PCB, LED, resistors, capacitors, a microphone, and transistors. The procedure involves playing a reference video, recording the screen with a high‑speed camera (240 fps), and measuring the time difference between a visual cue (white square crossing “0”) and the LED illumination. The time difference T₂ − T₁ yields the audio‑video latency.

Summary – The article concludes that synchronizing audio and video in WebRTC is achieved by calculating a target delay based on RTP/NTP timestamps, applying the appropriate delays to audio playback and video rendering, and optionally measuring the resulting latency with the described hardware method.