How Alibaba’s AI Powers Voice Ticketing and Facial Recognition in Shanghai Metro

Voice Ticketing Machine

Alibaba, Ant Financial and Shanghai Metro jointly launched a new generation voice ticketing machine that allows passengers to buy tickets by speaking natural language, even in noisy subway environments. The system supports direct station name or price queries and fuzzy destination search, with payment via Alipay QR code.

According to iDST AI expert Yan Zhijie, it is the world’s first far‑field voice interaction product that works accurately in strong public‑place noise, using a multimodal “speech + vision” solution that detects passengers approaching the machine and initiates interaction without a wake‑up word.

The solution consists of four subsystems: a large‑array microphone for sound source localization and enhancement; a computer‑vision module for face, eye and lip detection; a multimodal fusion engine that combines audio and video; and far‑field speech recognition, semantic understanding, dialogue management and speech synthesis.

Key technical highlights include high‑accuracy far‑field recognition in noisy stations, wake‑word‑free interaction, and a dialogue system that understands colloquial speech and continuously learns from real passenger conversations.

Smart Passenger Flow Analysis

The AI‑driven passenger‑flow analysis system uses video‑based human detection, machine‑learning data analysis and Alibaba Cloud DataV visualization to monitor crowd density, predict congestion, and support emergency response by estimating passenger numbers on platforms and trains.

Technical challenges addressed are robust human detection with existing low‑resolution cameras under varying lighting, de‑duplication of the same passenger across multiple cameras, and ultra‑short‑term flow prediction.

Facial‑Recognition Turnstile

Shanghai Metro’s facial‑recognition turnstiles employ Alibaba’s face detection, tracking and verification algorithms, achieving >99.5% accuracy on LFW and >95% recognition in 1:3000 scenarios, with sub‑200 ms latency on an Intel i3 PC.

The system leverages Alibaba Cloud’s distributed computing and storage, offering high concurrency, speed, precision and anti‑spoofing capabilities.