0 views collected around this technical thread.
This article explains how modern AI advances have transformed audio processing, covering digital audio fundamentals, automatic speech recognition (ASR), text‑to‑speech (TTS), voice cloning techniques, and provides practical Python code examples using OpenAI Whisper and HuggingFace TTS models.
This article provides a comprehensive overview of audio and video fundamentals, including signal conversion, PCM encoding, compression techniques, spatial audio concepts, video encoding standards such as H.264/H.265, streaming protocols, bitrate control, and practical optimization algorithms for both audio and video pipelines.
This article details how Huaisen reshapes the karaoke workflow by introducing innovative features such as clear‑singing pitch‑finding, a comprehensive editing SDK, and intelligent pitch‑correction algorithms, explaining the underlying audio analysis, strategy generation, and system architecture that enhance user experience across recording, editing, and publishing stages.
This article provides a comprehensive technical analysis of the music bullet feature in a K‑song community, detailing its core roles, client‑side production and consumption pipelines, real‑time mixing, alignment, volume balancing, precise seeking, performance optimizations, scalability, and sharing mechanisms across iOS and Android platforms.
This article examines Huaisen, a Kuaishou‑incubated karaoke and music video app, analyzing the K‑song market’s size and challenges from short‑video platforms, and detailing Huaisen’s innovative features such as music bullet comments and a reshaped karaoke workflow to revitalize the offline K‑song community.
This article introduces the Python library librosa, outlines its main audio processing features such as loading, visualization, MFCC, pitch detection, chromagram, and rhythm analysis, and provides complete code examples for each operation.
pydub is a Python library for audio processing that enables editing, converting, and manipulating audio files through integration with ffmpeg, supporting formats like MP3 and WAV.
The article details how Quanjian K‑Song has built a comprehensive mobile‑focused audio recording system since 2014, covering low‑latency capture, high‑quality sampling, lyric and vocal‑accompaniment alignment, ear‑return, pitch shifting, vocal enhancement, 3A processing, and AI‑driven scoring to deliver a professional karaoke experience on smartphones.
Quanmin K Song’s decade‑long mobile‑recording platform combines 48 kHz/16‑bit dry‑signal capture, sub‑70 ms latency via OpenSL ES/AAudio, real‑time clipping and noise detection, lyric‑ and vocal‑accompaniment alignment, pitch‑shifting, adaptive vocal enhancement, 3A DSP/AI processing, and AI‑driven pitch correction to deliver industry‑leading high‑quality mobile singing experiences.
The Kuaishou audio team secured first place in both the ICASSP 2024 Speech Signal Improvement and Audio Deep Packet Loss Concealment challenges by deploying a two‑stage GAN‑based speech enhancement system and a hybrid time‑frequency packet‑loss concealment model that dramatically improve real‑time communication quality.
The case study details how the Cloud Editing team tackled severe waveform loading delays, zoom lag, and inefficient IndexedDB storage by refactoring the processing pipeline, standardizing multi‑transaction storage, adding monitoring and cleanup tools, and rigorously testing releases, ultimately cutting processing times by over half and dramatically improving user experience.
The article explains WebRTC’s audio pipeline—from device capture through hardware or software 3A (AEC, ANS, AGC), Opus codec selection, and NetEQ jitter‑buffer handling—detailing how device specifics and scenario‑based configurations (live streaming, meetings, education, watch‑together) affect quality and why pure‑software 3A is emerging as the preferred future solution.
PaddlePaddle 2.4 introduces 167 new APIs—including sparse computing (paddle.sparse), graph learning (paddle.geometric), and audio processing (paddle.audio) modules—enabling efficient sparse model training and inference, graph message‑passing, advanced audio feature extraction, plus fresh loss functions, tensor utilities, and expanded vision transforms.
This talk presents the application of voice analysis in financial risk control, covering voice‑based risk feature extraction, single‑channel speech separation techniques, and speech‑text labeling methods, demonstrating how acoustic and textual cues can be leveraged to improve risk detection and model performance.
This tutorial explains how to create a lightweight speech synthesis tool using iFlytek's WebAPI, covering required environment setup, API credential acquisition, GUI design with Tkinter, and detailed Python code for WebSocket communication, audio handling, and WAV file generation.
ReolAudio is a lightweight, JavaScript‑based library that replaces memory‑heavy AudioBuffer editing with streaming and random‑access decoding, frame‑based data structures, and a high‑performance AudioWorklet player, dramatically improving memory usage, start‑up time, and waveform rendering for long audio projects.
This talk presents an overview of text‑plus‑speech multimodal emotion analysis, covering background, single‑modal text and audio models, the MSCNN‑SPU multimodal architecture, domain‑adaptation techniques, and future directions, with detailed model explanations, experimental results, and practical deployment insights.
This article presents an overview of speech translation, discusses its motivations and applications at ByteDance, compares cascade and end‑to‑end modeling approaches, introduces advanced encoder and decoder designs such as LUT, Chimera, and COSTT, outlines progressive multi‑task training and data‑augmentation strategies, and shares experimental results and Q&A.
This article describes how NetEase Cloud Music leveraged Serverless function computing to redesign its audio/video algorithm processing platform, covering the existing challenges, the selection criteria for Serverless solutions, the implementation details, performance gains, cost savings, and future directions.
The iQIYI M2VoC competition at ICASSP 2021, the first low‑resource multi‑speaker, multi‑style voice‑cloning challenge, attracted 153 academic and industry teams to tackle few‑shot (100 utterances) and extreme few‑shot (5 utterances) tracks, evaluated by professional listeners, yielding strong innovations and applications while confirming that single‑sample cloning remains unsolved.