How We Built a Scalable Mobile UI Automation Platform with Airtest and Jenkins

Background

With the massive adoption of mobile devices, mobile applications have proliferated, and users demand extreme convenience and experience, making mobile development and testing increasingly important. In the logistics industry, the company provides several mobile apps, such as the "掌中通" app, which displays work assignments, routes, client information, and salary settlement for couriers. The app’s accuracy and stability directly affect the company’s delivery progress, so effective automation tools are essential for quality assurance.

1. Mobile UI Automation Exploration (Current Situation)

The testing department has been exploring mobile automation for years. Using the BDD‑based Robot Framework together with Appium, they built keyword‑driven automation scripts for all key projects. Most testers can develop scripts in the IDE. However, as apps become more complex and business requirements change rapidly, the existing approach is hard to maintain.

Key problems to solve:

Rapid script development to keep up with fast business iterations.

Ability to run tests on dozens of devices simultaneously, execute monkey scripts, and quickly detect compatibility issues.

Unified management of test scripts and cases for fast regression before app releases.

Example scripts created with the original framework:

2. Automation Tool Upgrade (Evaluation & Selection)

Among many mobile automation tools, only a few support both Android and iOS. The most common are Appium and NetEase’s Airtest. Although both provide IDE support, Airtest is far more efficient. Its main advantages are:

Image‑recognition based scripting enables fast script creation.

Built‑in recording feature produces high‑quality scripts.

An example of a recorded script with strong readability:

3. Real‑Device Platform (Foundation)

Based on a second‑level development of atxserver2 and integrated with the company’s SSO authentication, a real‑device platform was built. Testers and developers can view available phones, detailed device information, borrow devices to reproduce issues, and verify script stability.

4. Mobile Automation Platform (Implementation)

With the above foundation, a mobile automation platform was designed with the following architecture:

Key features:

Jenkins pipelines run scripts; each task executes in a Jenkins slave container to maximize concurrency.

Scripts can target specific devices or run on all connected phones.

Task scheduling ensures a device runs only one script at a time; queued scripts wait for the device to become idle, and priority ordering favors important projects.

Unified script configuration allows manual execution or scheduled jobs.

After execution, the system aggregates results from all devices and presents a unified report.

Thanks to Airtest’s screenshot capability, each step is captured on every device, and video recording can be enabled. Errors are clearly visible in the recorded media.

The platform is now live, and key projects are gradually being integrated.

5. Future Outlook

Emerging technologies such as image recognition, 5G, and artificial intelligence open new possibilities for testing. For example, using step screenshots combined with image‑recognition can automatically detect differences across devices for compatibility testing.

We will continue to embrace technology, innovate, and improve testing efficiency and quality.