How Has Speech Recognition Evolved from Traditional Methods to Modern Deep Learning?

Introduction

Auditory perception is a fundamental element of artificial intelligence and a crucial entry point for information interaction. Automatic Speech Recognition (ASR) aims to convert spoken language audio signals into corresponding word sequences. Recent advances in deep learning have dramatically improved recognition performance, enabling high‑accuracy human‑machine interaction, speech translation, speaker verification, and more when combined with natural language processing.

Problem

What changes have occurred from traditional methods to the current mainstream approaches in speech recognition tasks?

Analysis and Answer

The speech recognition system consists of an encoder and a decoder. The encoder handles signal processing and feature extraction, while the decoder comprises search algorithms, acoustic models, and language models. The overall workflow is shown in Figure 1.

Signal Processing & Feature Extraction: Input audio is denoised, enhanced, and transformed into time‑frequency representations to encode speech signals.

Acoustic Model: Generates acoustic scores from extracted features, mapping speech features to phonemes.

Language Model: Estimates the probability of word sequences, often using n‑gram models such as bigrams.

Search Algorithm: Combines acoustic and language scores to select the highest‑scoring word sequence as the recognition result.

Traditional Methods

From the 1980s to around 2012, traditional ASR relied on Mel‑Frequency Cepstral Coefficients (MFCC) for feature extraction and Gaussian Mixture Model‑Hidden Markov Model (GMM‑HMM) acoustic models, as illustrated in Figure 2.

The encoder performs frame slicing, high‑frequency boosting, windowing, and denoising before extracting MFCC features, which map linear spectra to the perceptual mel scale and then to cepstral coefficients, improving recognition rates. The GMM‑HMM acoustic model estimates probability distributions for each HMM state, while the language model uses statistical n‑gram probabilities to refine the final word sequence.

Modern Methods

With the rapid evolution of ASR applications, researchers replaced traditional modules with deep neural networks (DNN). DNN‑HMM systems outperformed GMM‑HMM, leading to hybrid architectures (Figure 3). Subsequently, recurrent neural networks, especially LSTM‑based models, addressed temporal dependencies, enabling Connectionist Temporal Classification (CTC) to reduce word error rates (Figure 4). End‑to‑end models such as EESEN (CTC‑based) and attention‑based Listen, Attend and Spell (LAS) further eliminated separate language‑model and dictionary components.

References

[1] Hinton G, Deng L, Yu D, et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 2012, 29.

[2] Graves A, Fernández S, Gómez F, et al. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. Proceedings of the 23rd International Conference on Machine Learning, 2006: 369–376.

[3] Miao Y, Gowayyed M, Metze F. EESEN: End‑to‑end speech recognition using deep RNN models and WFST‑based decoding. 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), 2015: 167–174.

[4] Chan W, Jaitly N, Le Q, et al. Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016: 4960–4964.