How NetEase’s Saturn Platform Revolutionizes Mobile UI Automation

Background

In 2021 NetEase Cloud Music released a new version every two weeks. Integration testing required two days, most of which were spent on manual regression. Existing UI automation frameworks (Appium, SmartAuto) suffered from high maintenance cost, unstable UI element detection, delayed case entry, cross‑platform consistency problems, and poor root‑cause analysis.

Pain Points

Manual regression was inefficient; UI frameworks required frequent maintenance and produced unstable element locators.

Automation relied on manual device scheduling and management.

Smoke testing lacked an automated acceptance mechanism and depended on offline communication.

No unified platform existed for scheduling, managing, and storing automated test results.

Saturn Platform Overview

Late 2021 the front‑end QA and public‑tech teams built the Saturn platform. Core functions include:

Case recording – captures mobile actions, control information, embedded points, and screenshots.

Case management – uploads recorded cases, categorises them, and groups them by module or execution set.

Execution sets – defines collections of cases for quick execution and planning.

Device scheduling – supports private deployment, test‑type based scheduling, cross‑app scheduling, specific‑device allocation, and random device assignment.

Case execution – distributes cases to provider nodes, which invoke Athena or NETestWD for automation.

Reporting – aggregates execution steps, screenshots, manual check marks, failure categorisation, video recordings, and logs.

Device management – handles device lifecycle, purpose, and deployment categories.

Specialised modules – UI automation, React‑Native automation, startup performance testing, stability testing, coverage installation, etc.

Architecture

Platform side – user interaction, device management and scheduling.

Android provider – runs on macOS or PC, monitors Android devices, communicates with the platform, and dispatches automation tasks.

Athena app – records and replays automation cases, reports case data and execution results.

Android built‑in SDK – assists Athena in recording/replaying and prepares the execution environment (login, mock, backend).

iOS provider – runs on macOS, monitors iOS devices, maintains WebDriverAgent (WDA) and NETestWD stability.

NETestWD – launches the target app, receives cases, and reports execution results.

WDA (WebDriverAgent) – handles system dialogs on iOS devices.

iOS built‑in SDK – records automation cases, prepares the execution environment, and runs cases.

Device Scheduling Workflow

When a user triggers a case, the backend stores the request in a history table and pushes a SystemHistory message to a queue. Consumer threads read the message, generate device‑specific messages, and enqueue them. Provider threads listen for these messages, lock them, check for available devices, and either release the message (if no device is free) or dispatch it via WebSocket to the appropriate provider. The provider executes the case using Athena/NETestWD, reports the result back to the platform, which updates the case status and releases the device.

Device Management Page

Athena Workflow

Recording

Athena listens to ADB events, captures node information via Accessibility, and logs operation nodes, actions, and embedded points.

Playback

The built‑in SDK quickly prepares the environment, eliminating many pre‑steps.

If embedded points exist, Athena uses them to locate target nodes; otherwise it scores multiple search strategies to choose the best node.

Athena automatically handles system dialogs to avoid interference.

iOS SDK Workflow

Recording

Touch and scroll events are captured via AOP by intercepting UIControl.sendAction and UIView.addGestureRecognizer, and hooking UIScrollViewDelegate for scroll messages.

Keyboard input and assertions lack natural hooks, so a manual interaction tool is used.

Playback

The core is UIApplication.sendEvent; a synthetic UIEvent is constructed and dispatched through this method.

Fast environment preparation removes many pre‑steps.

If embedded points are unavailable, the system falls back to XPath‑based node location.

Case Storage

Test cases are stored as JSON expressions that contain execution‑environment details, mock interfaces, credentials, and ordered step sequences.

Device Farm

Applications in Cloud Music

UI Automation

Cases can be triggered manually or on a schedule and integrated with the CI pipeline. On a single device more than 30 cases run per hour (including up to two retries for failures). Screenshots and videos are attached to each execution.

Compatibility Testing

Compatibility cases automatically select devices from each OS version for testing.

Stability Testing

Both platforms support scheduled and manual triggers; the underlying engine uses the open‑source Fastbot project.

Startup Performance Automation

Scheduled builds launch automated tests that collect startup metrics via embedded points, visualise trends, and prevent performance regression.

React‑Native Automation

Image‑based comparison validates RN page rendering.

Results in Cloud Music

Learning curve reduced – any team member can record test cases.

Android P0 case coverage 72.95 % with a success rate of 91.87 %.

iOS P0 case coverage 71.86 % with a success rate of 91.33 %.

Case creation and maintenance efficiency improved 9× over Appium and 3× over SmartAuto.

Execution efficiency increased by over 1.5×.

Success rate doubled and remained above 90 %.

Problem‑location time shortened from days to hours.

Regression cycle reduced by 0.5 day.

Cross‑platform cases with shared embedded points are reusable.

Case JSON can be quickly updated for UI changes.

More than 2,000 automated cases recorded on the platform.

Over 200,000 automated executions per year.

Dozens of functional and stability issues discovered.

References

https://github.com/appium/WebDriverAgent

https://github.com/alipay/SoloPi

https://github.com/openatx/atxserver2

https://github.com/bytedance/Fastbot_iOS

https://github.com/bytedance/Fastbot_Android

https://github.com/alibaba/tidevice