Why WebRTC Latency Isn’t About the API: Go, ICE, DTLS, and Scaling

The fundamental bottlenecks of real‑time audio/video systems are the network model, congestion control, memory management, and concurrency scheduling—not the API surface.

Key Questions for Low‑Latency Design

Before building a low‑latency solution, answer these three mechanism‑level questions:

Why does WebRTC prefer UDP?

Why can ICE negotiation take 2–10 seconds?

Why must RTP packets be handled without arbitrary copying?

Without concrete answers, high‑concurrency deployments will become unstable.

1. Real WebRTC Layering

The practical protocol stack is:

Application
 ↓
Signaling (custom)
 ↓
SDP (Session Description Protocol)
 ↓
ICE (Interactive Connectivity Establishment)
 ↓
DTLS (Datagram Transport Layer Security)
 ↓
SRTP (Secure Real‑time Transport Protocol)
 ↓
UDP
 ↓
IP

Critical mechanisms to master:

ICE connection‑state machine

DTLS handshake flow

SRTP key derivation

RTP/RTCP feedback loops

Congestion‑control algorithms (GCC, TWCC)

2. End‑to‑End Latency Composition

Latency is the sum of several stages, not just network RTT:

Capture latency
+ Encoding latency
+ Packetization latency
+ Network transmission
+ Jitter buffer delay
+ Decoding latency
+ Rendering latency

WebRTC can stay under 500 ms because:

UDP eliminates head‑of‑line blocking.

No TCP retransmission wait.

RTP tolerates moderate packet loss.

Jitter buffer adapts dynamically.

Google Congestion Control (GCC) adjusts bandwidth in real time.

3. ICE Deep Dive

ICE operates as a state machine:

Gather candidates (host, srflx, relay).

Form candidate pairs.

Sort pairs by priority.

Perform connectivity checks sequentially.

Select the nominated pair.

State diagram:

new → checking → connected → completed
                         ↓
                       failed

Typical reasons for being stuck in checking :

No srflx candidate (STUN server unreachable or mis‑configured).

Both peers behind symmetric NAT.

UDP blocked by corporate firewalls.

Production systems should deploy TURN relays in addition to STUN to guarantee connectivity.

4. DTLS + SRTP Secure Link

WebRTC enforces encryption through the following flow:

Exchange DTLS fingerprint in SDP.

Establish DTLS handshake over UDP.

Derive SRTP keys from the DTLS session.

Transmit media using SRTP.

Because DTLS runs TLS on top of UDP, handshake packets may be lost and must be retransmitted; the handshake duration directly influences first‑frame latency.

5. RTP/RTCP Feedback Mechanisms

RTP carries the media payload, while RTCP provides essential control information:

NACK – request retransmission of lost packets.

PLI – request a new key frame (useful for video).

REMB – receiver‑estimated maximum bitrate for bandwidth adaptation.

TWCC – transport‑wide congestion control feedback.

Disabling RTCP causes rapid quality collapse when packet loss exceeds ~3 %.

6. Go Runtime for High‑Concurrency Media

Go is well‑suited for a WebRTC server because:

G‑M‑P scheduler maps goroutines efficiently onto OS threads.

Network I/O uses epoll/kqueue for scalable event handling.

Goroutine context switches are extremely cheap.

No explicit thread‑pool management is required.

Typical architecture: each PeerConnection runs in its own goroutine, allowing tens of thousands of concurrent connections. Open‑source SFU projects such as LiveKit and ion‑sfu (both built on the pure‑Go Pion library – https://github.com/pion/webrtc) demonstrate this model.

7. Memory Model – Avoiding GC Bottlenecks

Audio‑video pipelines process far higher data rates than typical web services. Example: 720p @ 30 fps ≈ 1 Mbps per stream → 1 Gbps for 1 000 connections. Copying or appending each RTP packet triggers heavy garbage‑collection pressure.

Optimization strategies:

Zero‑copy transmission (e.g., reuse the same buffer for send).

Maintain a pool of reusable byte slices.

Prevent interface{} escape to the heap.

Control slice growth to avoid frequent reallocations.

Use escape analysis to verify allocations:

go build -gcflags="-m"

8. Theoretical Limits of Pure P2P

In an N‑person conference, P2P requires O(N²) connections. The total number of unidirectional media streams is N × (N‑1). When N > 6, most clients hit upstream bandwidth limits, making pure P2P non‑scalable.

9. SFU Engineering Essentials

An SFU (Selective Forwarding Unit) forwards RTP packets without decoding or re‑encoding, merely rewriting SSRC identifiers. This reduces complexity from O(N²) to O(N): each client maintains a single connection to the SFU, sending one upstream stream and receiving N‑1 downstream streams.

Remaining engineering challenges:

Bandwidth layering (Simulcast).

Support for Scalable Video Coding (SVC).

Downstream link‑selection algorithms.

Coordinated congestion‑control across all forwarded streams.

10. High‑Concurrency Tuning Checklist

Increase UDP socket buffers: net.core.rmem_max / net.core.wmem_max.

Set CPU affinity to bind worker threads to specific cores.

Consider NUMA effects and allocate memory local to the processing cores.

Monitor and minimize Go GC pause times.

Track P99 latency to ensure tail‑latency stays within target.

11. Architecture Evolution Path

P2P prototype.

Single‑node SFU.

Multi‑node distributed SFU cluster.

Cross‑region routing and load balancing.

Edge‑computing offload for latency‑critical paths.

The real challenge lies in scheduling, network understanding, and overall system design rather than the WebRTC protocol itself.

12. Conclusion

WebRTC provides a complete, encrypted real‑time transport stack. Go offers a lightweight, high‑concurrency runtime. Combining the two yields a controllable protocol stack, tunable runtime behavior, and scalable distributed architecture—key to breaking the 500 ms latency ceiling.