From Human‑View Video to AI‑Understanding: Peking University’s Artic Framework Boosts Real‑Time AI Video Assistants

Real‑time video communication has traditionally optimized for human visual experience—clarity, smoothness, and low stall—using metrics such as PSNR, SSIM, and VMAF. With AI video assistants, the goal shifts to enabling large multimodal models to answer questions accurately and promptly, making the quality of specific visual regions far more critical than overall picture fidelity.

The paper "Artic: AI‑oriented Real‑time Communication for MLLM Video Assistant" (Wu et al., SIGCOMM 2026, https://arxiv.org/abs/2602.12641) proposes Artic, a system that incorporates the model’s perception state into the communication loop. Artic consists of three modules:

ReCapABR : Uses the model’s response confidence to decide when additional bitrate no longer improves answer accuracy, deliberately limiting bitrate to preserve bandwidth for future fluctuations.

ZeCoStream : Relies on the model’s feedback about which regions are essential for the current query, dynamically adjusting encoder quantization parameters to allocate more bits to those regions while reducing bits for irrelevant background. It also predicts future important regions to pre‑emptively protect them.

DeViBench : A benchmark that pairs high‑bitrate videos with degraded versions and selects question‑answer pairs that are sensitive to video quality, enabling quantitative evaluation of how transmission degradation impacts model understanding.

Experiments implemented in C++/Python compare Artic against standard WebRTC and various component combinations. Component‑level tests show ReCapABR reduces average latency by up to 148 ms under frequent bandwidth swings, while ZeCoStream raises answer accuracy from 0.39 to 0.60 at 290 Kbps and cuts the bitrate needed for 0.9 accuracy from 3171 Kbps to 908 Kbps. End‑to‑end evaluations on real mobile uplink traces demonstrate a 15.12% accuracy improvement and a 135.31 ms latency reduction across both BBR and GCC congestion controllers.

Client‑side overhead remains lightweight, as both modules involve only simple numeric calculations. Server‑side cost rises from $0.3126/min to $0.3974/min (≈27% increase) due to extra model feedback calls, a trade‑off justified by the resulting stability and accuracy gains.

In conclusion, Artic shows that making AI‑driven video interaction feel “human‑like” requires treating the network as a first‑class participant in the AI inference loop, adapting transmission strategies to the model’s real‑time perception needs.