What Were the Key Speech AI Breakthroughs at Interspeech 2016?

In September 2016, the premier speech‑and‑signal‑processing conference Interspeech took place in San Francisco, USA, with several Alibaba speech‑technology experts in attendance.

Interspeech, together with ICASSP, is one of the two most important international meetings for the speech community. Researchers from academia and industry presented work covering automatic speech recognition (ASR), speech synthesis, speaker verification, language identification, speech enhancement, multimodal processing, and language modeling.

Speech Recognition

1. CTC and related techniques – The popularity of CTC‑based ASR waned slightly at this meeting. Notable papers included a Deep CNN + CTC model by Bengio’s student, Daniel Povey’s chain model extending CTC with lattice‑free MMI, and Google’s Lower Frame Rate (LFR) neural network acoustic models that achieve performance comparable to or better than CTC.

2. Deep CNN technology – Deep convolutional networks continued to improve ASR accuracy. Representative papers were Microsoft’s CNN‑Attention fusion and IBM’s studies on CNN discriminative training, time‑pooling, and batch normalization.

3. Other deep model innovations – Highway and residual networks appeared in several papers, enabling deeper models with better recognition results. Notable works include small‑footprint highway‑connected DNNs and multidimensional residual learning based on recurrent networks. iFLYTEK introduced Compact Feedforward Sequential Memory Networks (FSMN) inspired by FIR filters.

4. Far‑field speech recognition – Research on distant‑microphone ASR grew noticeably, reflecting IoT demand. Dominant approaches used neural networks for adaptive beamforming, multichannel fusion, and reduced computational complexity. Representative papers covered neural‑network adaptive beamforming, recurrent models for auditory attention, and integrated feature extraction for multichannel acoustic models.

Speech Synthesis

Google DeepMind announced WaveNet, a generative model that directly predicts raw audio waveforms using a PixelRNN‑like architecture. When trained on large datasets, WaveNet produces highly natural‑ sounding speech and narrows the gap to human audio.

Other Topics

Neural‑network language modeling remains a hot research area, though efficiency challenges limit deployment. Recent progress combines n‑gram interpolation with RNN‑LMs; Microsoft’s “Conversational Speech Recognition System” reduced Switchboard word error rate to 6.3% using an RNN‑LM. Speaker and language‑identification papers focused on I‑vector, PLDA improvements, and attention‑based innovations.