Deploying Speech AI Services Quickly with NVIDIA Riva

The article introduces NVIDIA Riva, an SDK for real‑time Speech AI services that leverages GPU acceleration and provides pre‑trained ASR and TTS models ready for deployment.

It details the latest updates to Chinese speech recognition models, including support for unified models, punctuation prediction, mixed Chinese‑English models, VAD, speaker diarization, and inverse text normalization.

Riva ASR supports state‑of‑the‑art models such as Citrinet, Conformer, and FastConformer, offering multilingual capabilities and customization through hot‑word boosting, pronunciation dictionaries, and inference‑time tweaks without retraining.

The ASR pipeline can be customized at three levels: client‑side inference (e.g., hot‑word insertion), deployment‑time settings (e.g., latency vs. throughput modes, pronunciation dictionary), and server‑side training (e.g., acoustic model fine‑tuning, language model adjustments).

Riva TTS is described with its pipeline—text normalization, G2P conversion, spectrogram synthesis, and vocoder—and supports FastPitch and HiFi‑GAN models for multiple languages. Customization can be done via SSML (adjusting pitch, rate, volume) or by fine‑tuning the underlying models using NeMo.

The Riva Quickstart tool enables rapid deployment: after registering an NGC account and ensuring GPU support, users run riva_init.sh , riva_start.sh , and riva_stop.sh to manage the server, then use client scripts like riva_asr_client or riva_tts_client for inference.

A Q&A section addresses common concerns, explaining Riva's relationship with Triton and TensorRT, its focus on Speech AI, compatibility with NeMo‑trained models, and how to handle custom models, low‑memory GPUs, and Chinese dialect support.