Applying Deep Learning to Predict Front-End Compatibility Bugs in Mobile Apps

Background

Traditional front‑end UI compatibility testing relies heavily on manual judgment, and existing automated test cases use simple assertions (e.g., checking for the presence of specific text), which limits their ability to discover bugs.

Problem‑Solving Approach

The paper proposes using deep learning, a branch of machine learning based on multi‑layer neural networks, to replace part of the manual process for detecting visual compatibility bugs, focusing on static bugs like control misplacement and text overlap.

Bug Type Analysis

Front‑end compatibility bugs are categorized into static and dynamic types; the study concentrates on static bugs, particularly the sub‑type of text overlap, to verify the feasibility of deep‑learning‑based prediction.

Sample Generation

Using 3,755 common Chinese characters, 31×31‑pixel images are generated on a white background: one image per character as a positive (unobstructed) sample and two images with artificial occlusion as negative samples, creating a balanced dataset of 20,000 test images and 120,000 training images.

Model Training

Image lists (test.txt, train.txt) are created and converted into LMDB databases; a CNN model based on a modified AlexNet architecture is defined in Caffe, and training is performed on a GPU (Tesla K40c) for 24,000 iterations, achieving about 99% accuracy in roughly 28 minutes.

Model Prediction

Screenshots of the app are captured, the target controls are extracted, and the trained model predicts whether the control suffers from the overlapping‑text bug, successfully validating the approach on real‑world cases.

Conclusion

The experiment confirms that deep learning can discover and predict a specific type of front‑end compatibility bug (text overlap) and suggests that the same pipeline could be adapted to other bug types with appropriate sample construction and network adjustments.

Author Introduction

Yin Fei, senior test engineer at Baidu Nuomi, has been with Baidu since 2011, working on quality assurance for PC browsers and client products, and currently focuses on testing the Baidu Nuomi merchant app, with deep expertise in mobile testing tools.