Ctrip's Mobile App Testing Infrastructure and Automation Platform as a Service (APS)

In recent years Ctrip's business has grown dramatically, with wireless traffic accounting for about 70% of total bookings and the mobile app exceeding 700 million downloads by mid‑2015, creating an urgent need to improve app testing efficiency and quality.

The company faced several challenges in wireless app automation: lack of mature open‑source frameworks for multi‑platform support, the inadequacy of Selenium Grid for mobile testing, and numerous device‑specific permission dialogs such as GPS prompts and charging notifications.

To address these issues Ctrip built a mobile testing infrastructure. Real devices were chosen over emulators because emulators are slow on HDDs and cannot fully replicate hardware behavior. Device types were selected based on the most frequently used models in the Ctrip app, ensuring coverage for the majority of users.

Device scheduling uses a Master/Slave model managed by Jenkins CI, where each Jenkins job runs on a slave that hosts two (or more) devices, enabling parallel execution without the overhead of reinstalling the app for each test run. Selenium Grid was evaluated but discarded due to its limitations; instead, a custom Appium broker approach was adopted.

Device monitoring is handled by the open‑source Smartphone Test Farm, extended with custom features to detect Wi‑Fi disconnections, full‑charge states, and other conditions that could affect test stability. By October 2015 the lab contained over 100 devices.

The Automation Platform as a Service (APS) evolved from a simple automation framework into a full lifecycle platform that supports project initialization, test scheduling, execution, reporting, and monitoring. The core framework is built on TestNG and Appium, handling pre‑processing, device initialization, parallel test execution, environment cleanup, and report generation.

Special handling is required for devices with customized Android builds that present unexpected permission dialogs (e.g., Meizu GPS prompts, Xiaomi install confirmations). APS pre‑processes such devices by first installing the app via ADB and then launching it with Appium.

Because Appium does not natively support parallel execution, APS assigns distinct ports for each instance and modifies shared file access in the Appium source when necessary.

Test results and logs from all devices are aggregated in a database and displayed on a portal for unified reporting.

APS also simplifies CI integration, test case scheduling, execution, and monitoring, providing a transparent experience for users and reducing the learning curve.

With the infrastructure and APS in place, Ctrip introduced a suite of automated products, including a continuous integration pipeline that builds and tests the app every two weeks across more than ten business units, a smoke‑test suite that checks page rendering using element counts and image recognition, and specialized tests covering installation, launch performance, crash detection, and resource usage.

These efforts have enabled Ctrip to detect and resolve dozens of issues per release cycle, improving app stability and user experience.

Author: Liu Lifeng, Senior Test Manager at Ctrip