How AI Is Transforming Real‑Time Audio: Insights from LiveVideoStackCon 2020

From October 31 to November 1, 2020, LiveVideoStackCon 2020 was held in Beijing under the theme “Multimedia Opens a New Vision”, covering audio, video, and image technologies across many industries.

Artificial‑intelligence topics were especially popular. Dr. Hao Yiya, senior audio algorithm engineer at NetEase Cloud, delivered a keynote titled “Productizing AI Audio Algorithms in RTC” and shared practical AI applications in a round‑table.

She explained that AI, driven by CNN, RNN and other deep‑learning models, is increasingly used for audio denoising, scene classification, echo control and blind source separation.

However, AI adoption in real‑time communication (RTC) audio faces three main challenges: computational complexity, limited generalization ability, and robustness under diverse scenarios.

Because of these constraints, AI has not yet replaced traditional signal‑processing methods. Dr. Hao suggested a modular approach: split end‑to‑end processing into separate AI modules, allowing each to be optimized and handled by specialized experts.

Using AI‑based noise‑estimation as an example, a lightweight deep‑learning model can replace the traditional noise‑estimation module, offering a simple training target and better suitability for DNNs.

Looking ahead, she expects more advanced neural‑network architectures and efficient GPUs to drive AI integration in RTC, while massive data collection remains essential. NetEase Cloud already maintains a full data‑collection pipeline and has applied AI to audio denoising and video super‑resolution.

Since 2015, NetEase Cloud has focused on instant‑messaging and audio‑video technologies, serving over one million developers, handling 10 trillion messages, and reaching 300 million daily active users. The company continues to innovate by embedding AI into its upgraded audio‑video calling product.