How Alipay’s xNN Brings Deep Learning to Millions of Mobile Devices

Deep Learning – Cloud or Mobile?

Recent breakthroughs in deep learning (DL) for image, speech, and language tasks have created a perception that DL requires heavy computation and large models, often pushing data collection to the cloud. However, for many applications this cloud‑centric approach is a compromise rather than a necessity.

During the "Scan Five Blessings" campaign, Alipay deployed thousands of servers for image‑recognition models, yet still faced overload and had to fall back to a simpler visual algorithm, degrading user experience. Sending data to the cloud also incurs latency, bandwidth, and privacy concerns, making on‑device DL a genuine need for many scenarios.

Two Major Challenges

Although mobile processors have become more powerful and model‑lightening techniques have advanced, deploying DL on a national‑scale app like Alipay faces two strict constraints:

Huge device diversity: the app must run smoothly on billions of devices, requiring a universally compatible, low‑memory, fast solution.

Severe package‑size limits: the app’s binary is already packed with features, so any new model must be extremely small or delivered dynamically without harming the user experience.

Five Technical Goals

Alipay’s xNN was built to meet the following objectives:

Light models : aggressive compression while preserving accuracy.

Small engine : a highly trimmed mobile SDK.

Speed : joint algorithm‑ and instruction‑level optimizations for faster inference.

Universality : optimized for common CPUs rather than high‑end GPUs, supporting CNN/DNN, SSD, RNN, LSTM, etc.

Ease of use : a toolchain that lets algorithm engineers convert and deploy cloud models to mobile without deep expertise in compression or mobile development.

Main Features Overview

xNN provides a full lifecycle for DL models—from compression to deployment to runtime monitoring. It consists of a backend (xqueeze toolchain) and a frontend deployment framework.

The backend supports multiple training frameworks; developers can compress and optimize models with xqueeze, dramatically reducing size and increasing speed. Compressed models can be bundled in the app or delivered on‑demand.

The frontend offers efficient forward‑prediction, model delivery, data statistics, error reporting, and a JavaScript API for seamless H5 integration, enabling dynamic model updates without app releases.

xqueeze Model Compression

The xNN‑xqueeze pipeline includes five steps: neuron pruning, synapse pruning, quantization, network transform, and adaptive Huffman coding. The first three are lossy but are fine‑tuned to keep accuracy loss negligible; the last two are loss‑free. This process not only shrinks model size but also sparsifies the network, boosting inference speed.

Compared with classic DeepCompression, xqueeze adds advanced neuron pruning and network‑transform capabilities, automatically merging or replacing layers and applying adaptive Huffman coding to achieve higher compression ratios.

For a business‑classification model, xqueeze achieved a 45.5× compression with only minimal accuracy loss, surpassing traditional methods by 60%.

xNN Computation Performance Optimization

Performance gains stem from both algorithmic and instruction‑level improvements. Pruning creates sparse weights, which xNN’s sparse‑operation modules skip during convolution and fully‑connected calculations, reducing compute cost.

Network‑transform optimizations combine equivalent layers to cut redundant work, while the instruction layer provides hand‑written assembly to efficiently schedule kernels, improve cache usage, and balance multi‑threaded loads.

On a Qualcomm 820 CPU, xNN delivers 29.4 FPS for a SqueezeNet‑based model; on an Apple A10 CPU (iPhone 7) it reaches 52.6 FPS, outperforming Core ML’s CPU‑GPU hybrid approach.

Real‑World Deployment

Alipay has integrated xNN across the app. Over 90 % of Android and iOS devices use xNN for front‑camera object classification in the "AR Scan" feature, enabling functions like "AR flower identification". The latest model is compressed below 100 KB, and the full SDK adds only ~200 KB to the app; a trimmed version can be reduced to just over 100 KB.

Since its launch, xNN has sparked strong internal interest, with many mobile DL projects now built on the engine and slated for user release in the coming months.