Applying Deep Learning and AI on Mobile: Baidu App Cases and Technical Insights

The Baidu App team has been dedicated to bringing artificial intelligence to mobile platforms, achieving rapid development in recent years and redefining user experiences across many industries.

With the large‑scale rollout of 5G, increasingly powerful smartphones, and the rapid adoption of AIoT devices, there is still huge potential for cloud‑edge‑device AI architectures on mobile. Understanding mobile deep‑learning principles and how to apply them in real products is essential.

Mobile deep‑learning deployments can be categorized into two approaches:

1) Fully on‑device inference, which offers the best user experience with seamless, lag‑free interaction. Examples include Baidu’s “Pick Image” and the plant‑identification app “ShiHua”.

2) Server‑side inference with the mobile device only handling UI display, which is easier to implement and has lower development cost.

Several applications illustrate these approaches:

Plant and flower recognition – The “ShiHua” app, developed by Microsoft Research Asia, identifies flowers from a captured image and provides detailed information.

Stylized visual effects – Computer‑vision techniques enable filter effects in apps such as Philm, Prisma, and Artisto, applying deep‑learning‑based style transfer to images and videos.

Video subject detection – Used for identity verification and dynamic annotation. A 2017 demo showed real‑time detection of video subjects, enabling features like jumping to the first appearance of a specific actor or product.

Deploying deep learning on mobile faces many challenges, including model compression, compilation pruning, code size reduction, multi‑platform support, and assembly‑level optimizations. Overcoming these difficulties is critical for stable, efficient on‑device performance.

The article also details the development of an AR real‑time translation feature. The workflow includes:

• OCR text detection using deep‑learning models. • Text recognition (often with GRU‑based networks). • Translation either on‑device or via server request. • Tracking the original text position to correctly overlay translated text, handling camera movement and background color extraction.

After evaluating on‑device versus server‑side translation, the team chose server‑side results to avoid accuracy loss in long sentences, while still optimizing latency.

A technical flowchart and several screenshots illustrate the AR translation pipeline and its visual output.

Finally, the article promotes a book that shares practical experience in mobile deep‑learning, covering mathematical foundations, mobile hardware architecture, model compression, and performance tuning for engineers interested in mobile AI development.