How to Build Objective, Repeatable RTC Audio Quality Tests for Real‑Time Communication

Introduction

With the widespread adoption of 5G networks and the impact of the pandemic, real‑time audio/video (RTC) scenarios such as meetings, live streaming, online education, and tele‑medicine are increasing, raising the demand for high‑quality RTC audio. Building an objective, standard, and repeatable evaluation system is therefore urgent.

Ideal Communication Model

Face‑to‑face communication in a quiet laboratory yields the best results. The model highlights several key characteristics:

Quiet environment : background noise (NR15) is minimal, allowing listeners to focus on the target voice.

Suitable reverberation : excessive reverberation reduces intelligibility.

Clear, natural voice : balanced frequency, smooth speech, moderate speed.

Moderate volume : louder speech improves intelligibility without causing distortion.

Low latency : delays under 200 ms are imperceptible; face‑to‑face latency is about 3 ms.

RTC Audio Signal Path

Audio travels from the speaker through air, microphone, A/D conversion, enhancement processing (noise reduction, echo cancellation, volume control, dereverb), encoding, packetization, transmission, decoding, NetEQ, D/A conversion, and finally playback.

Common Audio Issues in RTC

Volume issues : silence, low volume, clipping, distortion, fluctuating levels.

Echo issues : leakage, residual echo, speech damage such as suppression or chopping.

Noise issues : residual, unstable background noise.

System‑introduced issues : hum, pop, other artifacts.

Quality issues : muffled, distorted, harsh, mechanical sounds.

Network issues : jitter, packet loss, fast/slow playback, mechanical noise.

Subjective Testing Methods

Early subjective tests used two‑person calls to simulate real usage, focusing on three dimensions:

Listening Quality : the quality perceived by the listener (single‑direction).

Talking Quality : the quality perceived by the speaker, affected by echo and side‑tone.

Conversation Quality : overall duplex communication, influenced by echo, double‑talk, and end‑to‑end delay.

Dimensions of Subjective Evaluation

Key aspects include timbre (frequency spectrum), quality (clarity, smoothness, naturalness), volume consistency, noise suppression, echo suppression, and latency.

Challenges of Subjective Testing

Low repeatability due to environmental and human variability.

Low efficiency because tests require continuous participation of two people.

Limited coverage of scenarios and device combinations.

Strong subjective bias from individual hearing perception and physiological state.

Acoustic Laboratory

NetEase Cloud‑Communication’s acoustic lab provides a controlled environment for objective testing. Core equipment includes:

Head‑shoulder simulator with mouth and ear models (IEC 60318‑4/ITU‑T Rec. P.57 Type 3.3).

Four high‑fidelity speakers creating a uniform sound field.

Multi‑channel sound cards supporting 8‑in‑8‑out capture and playback.

Four telecom interfaces for multi‑person and echo‑double‑talk testing.

Objective Testing Standards

Evaluation methods are classified as subjective vs. objective, reference vs. non‑reference, and perceptual vs. non‑perceptual. Perceptual objective metrics, such as PESQ and POLQA, model human hearing to predict subjective scores.

Automation and 3A Testing

The lab supports fully automated 3A (Audio, Echo, Noise) testing:

Cross‑platform support: Android, iOS, Windows, macOS, Web.

TC scripts automatically control network loss, delay, jitter, and bandwidth.

APIs automate meeting entry, profile switching, parameter control, and leaving.

Automatic collection of bitrate, packet loss, freeze, and other metrics.

One‑click execution generates baseline reports for version comparison.

Conclusion

By combining a professional acoustic lab, objective perceptual standards, and automated 3A testing, RTC audio quality can be evaluated reproducibly, supporting algorithm development, AI data collection, and rapid baseline comparisons across versions and competitors.