Real-Time Evaluation System for Adaptive Bitrate (ABR) Algorithms and Controlled Bitrate Distribution

This article introduces RESA, a real‑time evaluation system for multiple Adaptive Bitrate (ABR) algorithms developed by iQIYI. RESA continuously tests and compares ABR algorithms in live user environments, leveraging reinforcement‑learning techniques to provide suitable bitrate services for all devices.

The paper first outlines three major problems of existing ABR research: (1) evaluations are performed on open‑source datasets rather than real user networks; (2) QoE metrics are based on single‑user playback data and rely on empirically set parameters; (3) current ABR algorithms improve user QoE but increase the service provider’s bandwidth cost.

To address these issues, a two‑part system architecture is proposed: a client side that collects required data, receives bitrate decisions, and reports QoE metrics; and a server side that includes an algorithm proxy, algorithm implementation modules, and a QoE calculation module. Communication between client and server uses WebSocket + ProtoBuf for low‑latency data transfer.

The QoE scoring model combines three sub‑metrics—clarity, fluency, and smoothness—into a unified score. The model introduces adjustable parameters α, β, γ, which are derived from user viewing‑behavior preferences rather than expert experience. The chosen values are α=35, β=50, γ=1000.

Extensive online experiments evaluate classic ABR algorithms (BOLA, MPC, Pensieve) as well as three extreme baselines (MAX, MIN, ECHO). Results show that Pensieve achieves the highest overall QoE, followed by BOLA and MPC, while the improvement over the baseline ECHO mode is modest (≈12%). The evaluation also reveals that higher clarity often leads to increased bandwidth consumption; Pensieve raises high‑bitrate video duration by 41% and bandwidth usage by 23%.

To mitigate uncontrolled bandwidth growth, the paper proposes ABSbc (Adaptive Bitrate Streaming with Bitrate Control), a reinforcement‑learning‑based ABR algorithm that adds a bitrate‑control dimension. This dimension maps a control value in [0,1] to a target bitrate, allowing the system to steer the overall bitrate distribution according to operator‑defined preferences while keeping bandwidth cost under control.

The reward function of ABSbc distinguishes between selected bitrate lower than the target (penalty) and greater or equal to the target (reward), as illustrated by the provided formulas. Experiments demonstrate that varying the bitrate‑preference value directly influences the resulting bitrate distribution.

In summary, the RESA platform provides a reliable, real‑time framework for evaluating ABR algorithms across millions of users, introduces a comprehensive multi‑user QoE metric, and presents a controllable‑bitrate ABR solution that balances user experience with network cost.