Boosting Mobile App Quality: Shentong’s Custom DevOps Automation Framework

Background

Mobile has become a dominant feature of the Internet industry. After more than a decade of mobile Internet development, Shentong, a courier logistics enterprise, has increased its technical investment and built a relatively complete mobile product matrix, including the "ShenXing" app for couriers, a driver app, a station app, a transfer‑center app, a super‑merchant app, and various mini‑programs.

01 What to Test and How to Test

Functional Testing

Functions are divided into two categories:

Strongly dependent functions (e.g., parcel pickup/delivery, station inventory) require high availability.

Weakly dependent functions (e.g., management reports, data analysis) focus on data timeliness and consistency.

Manual testing dominates, but growing feature complexity demands automation for efficiency.

Compatibility Testing

Shentong’s mobile products run on native apps (Android, iOS) and H5‑based mini‑programs. Because testing every device model is infeasible, a "big‑catch‑small" strategy is used, prioritizing devices based on online user data. Cloud real‑device services and internal devices are combined, and group‑controlled devices plus automation are employed to speed up execution.

Specialized Testing (Performance/Network/Power/Stability)

Key scenarios such as offline operation, long‑duration scanning, and power consumption are tested with automated solutions to avoid repetitive manual effort.

02 How to Improve Quality and Efficiency

Two main pain points were identified:

R&D lacks an online packaging and release platform, causing time‑consuming offline builds.

Testing involves massive manual regression and specialized tests, consuming a tester’s full day for a major version.

The goal is a solution that balances quality and efficiency.

Quality Process

Shentong built a mobile‑focused DevOps pipeline that automates code merge, packaging, quality gate, security hardening, and release. This reduces daily packaging time from two hours to zero and enables online quality control.

The pipeline defines three stages:

Test stage : static code scanning and UI smoke automation ensure basic package quality.

Integration stage : regression, performance, and stability tests are accelerated using a mobile automation framework.

Release stage : flexible gray‑release (from single sites to whole provinces) with continuous feedback collection.

Automation Framework

The framework is built on Jenkins, integrating DevOps, GitLab, and a test‑management platform. Slave servers drive physical devices for test execution, and all test data are persisted for traceability and trend analysis.

Key features include:

Multiple trigger strategies (manual, scheduled, code‑change‑driven).

Support for UI automation, API automation, static scanning, and stability testing.

Distributed, concurrent, and group‑controlled execution to improve device utilization.

Persistent test data for result notification and quality metrics.

Remote access for on‑demand testing.

Automation Cases

Based on Appium, the framework adds extensive customizations to improve script authoring efficiency and stability, such as unified pre‑data construction, popup detection and auto‑close, camera/scan activation, page navigation control, and dynamic element location.

These enhancements have enabled:

Nearly a thousand packaging tasks.

Hundreds of automated test scenarios covering self‑test, smoke, full regression, and specialized tests.

Discovery of dozens of critical quality issues, saving test resources while enhancing user experience.

Next Phase Plans

Future work will focus on:

Continuously enriching iOS automation cases.

Integrating automation for WeChat mini‑programs.

Expanding the testing platform to include device management, tool integration, and online monitoring.

Exploring intelligent testing solutions to further boost efficiency and accuracy.

Due to space limits, many details are only outlined here; deeper discussion is welcomed.