Speech Recognition and Synthesis: Principles, Challenges, Optimizations, and Tencent Cloud Use Cases

Author Bio

Ni Jie, Senior Product Manager at Tencent Cloud, holds a master’s degree from Beijing University of Posts and Telecommunications and leads the AI Application Product Group, focusing on intelligent voice AI products with extensive experience in internet and finance sectors.

1. Speech Recognition Basics (ASR)

Speech recognition converts audio signals into text by extracting acoustic features, building acoustic models, and using dictionaries and language models to search and decode within a defined space.

1.1 Industry Speech Recognition Level and Challenges

In ideal conditions (quiet environment, close‑range, standard Mandarin, read speech) accuracy can reach 97%, but real‑world factors such as colloquial speech, mild accents, background noise, far‑field capture, and severe accents reduce accuracy to 85‑90% or lower.

Noise interference (e.g., car cabin echo)

Far‑field recognition

Domain‑specific vocabularies

Dialects and accents

Colloquial speech variations

High‑quality capture in noisy, multi‑speaker environments

2. Speech Synthesis (TTS)

Speech synthesis transforms text into natural‑sounding speech, enabling human‑machine interaction across many scenarios. Modern TTS, led by Google’s WaveNet, achieves MOS scores above 4.5, comparable to real recordings.

2.1 Speech Synthesis Challenges

Voice customization for brand identity

Recording duration and cost

Voice adaptability for different use cases

Handling polyphones and special pronunciations

Realism (accuracy, fluency, prosody)

Subjective quality assessment

3. Optimization in Typical Deployment Scenarios

3.1 Voice Input Method

Initially embedded in smartphones by manufacturers (Google, Apple, Samsung), voice input is now moving from standalone input‑method apps to integration within specific apps (e.g., in‑game voice chat).

3.2 Recording Transcription (Human‑to‑Human Interaction)

Used for quality control and compliance in call centers, transcription faces challenges such as varied speaking styles, background noise, and the need for high‑quality audio; best practices include proper engine parameters, high‑quality recording, and customized language models.

3.3 Customer Service Robots

Voice‑enabled robots handle repetitive customer service queries, combining ASR and TTS to provide multi‑channel support and improve user experience, turning AI‑driven voice systems into a competitive advantage.

4. Tencent Cloud Intelligent Voice Case Sharing

Tencent Cloud has deployed voice solutions in finance (voice‑controlled transfers), cultural heritage (audio guides for the Palace Museum), and hospitality (smart speaker “Xiaowei” in Atour hotels) to control devices, provide information, and enhance user interaction.

Additional applications include audio content moderation, courtroom transcription, and real‑time translation.