Optimizing NetEase Cloud Music Audio/Video Processing Platform with Serverless

NetEase Cloud Music originally handled audio and video processing for its music library using traditional server-based clusters, encountering issues such as long execution times, multi-language deployment complexities, and scaling challenges as the catalog grew to over 60 million tracks.

The team identified five common use cases for audio/video technology—bitrate switching, song recognition, chorus detection, pitch generation for karaoke, and automatic lyric romanization—highlighting the need for a unified, service-oriented processing platform.

After evaluating Serverless options, the team selected public cloud function computing (FC) because it matched their requirements for cost‑effective elasticity, simplified deployment via image delivery, and the ability to orchestrate algorithms without managing servers.

The migration focused on offloading compute‑intensive algorithm tasks to FC while keeping the overall input/output interfaces unchanged, adding traffic control and fallback to private cloud for high‑availability. Deployment leveraged CI/CD image pulling, monitoring integrated with both cloud alerts and internal systems, and code designed for hybrid‑cloud compatibility.

Initial results showed a ten‑fold speed increase for feature‑extraction algorithms, significant cost reductions for low‑frequency sparse algorithms, and faster cold‑start times thanks to FC’s image caching, allowing services to spin up within seconds.

Future plans include further Serverless adoption to reduce operational overhead, exploring GPU‑accelerated instances for more complex algorithms, and extending Serverless to real‑time audio/video processing scenarios to handle peak‑valley traffic patterns.

Author: Liao Xiangli, Head of Music Library R&D at NetEase Cloud Music (joined 2015).