How NetEase Cloud’s VQC Engine Balances Video Quality, Smoothness, and Latency

VQC’s Role in Video QoE

Video QoE comprises clarity, smoothness, and latency, jointly determined by network QoS, video processing algorithms, and the Video Quality Controller (VQC).

QoE Components

Network QoS: provides usable bandwidth.

Video processing algorithm: maximizes quality at a given bitrate.

VQC: controls bitrate for smoothness and latency, and balances clarity with smoothness.

NetEase Cloud VQC Implementation

The VQC design follows WebRTC’s module layout, consisting of four monitoring modules and a strategy module. Input parameters pass through the monitors, producing state results that the VQC strategy module uses to generate control signals for the video pipeline.

QualityScaller

Monitors current encoding quality, focusing on clarity and encoder stability. It receives QP thresholds, current frame QP, and frame‑drop statistics, then outputs a quality assessment.

QpSmoother uses exponential weighted accumulation to compute a statistical QP value: y = alpha * previous_y + (1 - alpha) * sample where sample is the current frame QP and alpha varies per encoder (e.g., 0.9995 for upper bound, 0.9999 for lower bound on OpenH264).

Final QP statistics are compared against encoder‑specific thresholds (e.g., 24–37 for OpenH264) to judge video quality.

OveruseFrameDetector

Detects whether the current performance can sustain the target frame rate, influencing video smoothness. It takes target frame rate, resolution, CPU‑usage thresholds, and frame‑capture/send times, outputting a “CPU good” or “CPU bad” state.

RateAllocator

Decides bitrate allocation in mixed‑stream scenarios, distributing limited bandwidth among large and small streams, merging streams when bandwidth is scarce, and reducing downstream bandwidth when needed.

MediaOptimization

Monitors and adjusts real‑time bitrate and frame rate to prevent network congestion. It uses a funnel‑style FrameDropper that decides whether to drop frames based on accumulated bitrate versus a capacity linked to acceptable latency.

VQC Decision Model

The decision module aggregates results from all sub‑modules and user‑defined scene settings to adjust video parameters (resolution, frame rate, simulcast selection, codec, etc.). It contains two independent state machines (video‑quality and performance) and a policy engine that sets thresholds, chooses adaptation steps, and triggers further adjustments when needed.

Optimization Strategies

Resolution adjustment

Frame‑rate adjustment

Simulcast stream selection

Pre‑processing algorithm toggles

Codec adaptation

VQC supports configurable scenarios (communication, live streaming) and automatically adapts step sizes and paths based on extensive testing.

QA Summary

Q: How does rate‑controller handle simulcast in SFU forwarding mode? A: The server uses a configurable TopN strategy, allocating high‑quality streams to top viewers and lower‑quality streams to users with poorer network conditions.

Q: How is codec switching achieved without renegotiation? A: A private capability‑negotiation protocol inside the channel enables seamless codec switches without SDP exchange.

Q: How are key‑frame requests balanced for new participants? A: Each new user sends an intra‑request, but the receiver controls the key‑frame interval to avoid excessive traffic.

Overall, the article details the design of NetEase Cloud’s VQC system and its practical adjustments to improve video QoE, acknowledging trade‑offs and the need for scenario‑specific tuning.