Python Tool for Converting English Videos to Chinese Dubbed Videos with Subtitles

This article provides a comprehensive guide on developing a Python tool to convert English videos into versions with Chinese dubbing and subtitles. The process is divided into seven main steps: extracting audio using ffmpeg, performing speech recognition with whisperx to extract subtitles, translating subtitles using googletrans or OpenAI, generating voiceovers with Microsoft TTS, replacing the original audio using ffmpeg, converting SRT subtitles to ASS format for better styling control, and finally adding ASS subtitles to the video using ffmpeg.

The detailed implementation includes code examples for each step. Step 1 demonstrates audio extraction using ffmpeg's subprocess module. Step 2 shows how to use whisperx for speech recognition to generate transcript files. Step 3 covers text translation using the googletrans library. Step 4 explains voiceover synthesis using Azure Cognitive Services' Speech SDK. Step 5 details audio replacement in the video file. Step 6 covers SRT to ASS format conversion using the pysubs2 library. Step 7 shows how to embed subtitles into the final video.

The article also provides a complete main program logic that orchestrates all these steps, including file path management and sequential processing. It concludes with important notes about package installation, environment configuration, and API key requirements for services like WhisperX, Google Translate, and Azure TTS.