How Meituan’s AlphaTest Revolutionizes Mobile UI Automation Across Multi‑App, Multi‑Tech Stacks

Background

Meituan Waimai’s platform supports multiple business lines (food delivery, flash‑sale, group‑buy, pharmacy, errand) and a heterogeneous front‑end stack: native code, Meituan’s Mach framework, React Native, Mini‑Programs and H5. Over two years the number of UI test cases grew from ~6,000 to >12,000 and the same test suite had to run on three apps (Waimai, Meituan, Dianping), causing the manual testing effort to become unsustainable.

Objectives

Usability : minimal learning curve, click‑through operation.

Platform coverage : iOS, Android and all Meituan tech stacks (Mach, H5, React Native, Mini‑Programs).

Stability : reliable execution without tool‑induced flakiness.

Maintenance cost : low effort to adapt to requirement changes or architecture refactoring.

Scalability : ability to extend when current capabilities fall short.

Tool Evaluation

Is building a custom tool just reinventing the wheel?

Three representative open‑source solutions were examined:

Appium : WebDriver‑based, supports iOS/Android/Windows, but UI element APIs differ across platforms, limiting cross‑platform script reuse.

Airtest Project : image‑recognition driven, easy to start, yet image matching accuracy is lower than human perception and fragile to resolution changes.

SoloPi : pure record‑replay for Android, cannot handle data‑driven scenarios or cross‑app reuse.

None satisfied all Meituan requirements, leading to the development of the in‑house AlphaTest platform.

AlphaTest Architecture

Command‑Based Recording

AlphaTest transforms each user interaction into a structured command rather than a monolithic script. A visual command editor lets users inspect and modify commands without writing code, dramatically reducing maintenance cost.

Pre‑condition Module

Users define environment prerequisites (account login, mock data, location, etc.) as separate steps. During playback the SDK automatically applies these steps, enabling one‑click environment simulation and reusable pre‑conditions.

Data Consistency

Before both recording and playback the app’s local cache is cleared. Network traffic generated during recording is captured and replayed, ensuring that the same data context is used for both phases and isolating UI failures from data‑related failures.

Multi‑Layer Locator

Element identification uses a three‑tier strategy:

ViewPath : rich view hierarchy attributes collected via SDK.

Image matching : fallback when ViewPath fails.

Coordinate fallback : final resort for static UI.

Gestures are issued at the element’s centre; for overlapping controls AlphaTest combines view‑based and coordinate‑based targeting to balance precision and stability.

Traceability

Each command execution records status, a screenshot, and any exception. After a run AlphaTest assembles a video‑recorded report together with aggregated logs, enabling one‑click root‑cause analysis.

Maintenance Scenarios

Major requirement change : full re‑recording.

Minor UI or data change : edit commands directly (e.g., modify network fields, ViewPath, assertions) without re‑recording.

Technical upgrades (architecture, UI redesign) : AlphaTest automatically repairs affected commands by re‑identifying elements with image + coordinate and regenerating ViewPaths.

Cross‑App Playback

App‑specific differences (initial pages, AB‑experiment configs, API mappings, scheme mappings) are stored in a configuration table. When playback detects a different target app, the platform applies the appropriate mappings, allowing a single test case to run on Waimai, Meituan and Dianping.

Event‑Tracking (埋点) Verification

AlphaTest can record event‑tracking logs during recording and compare them with logs captured during playback. Field‑level tolerance rules can be configured to ignore dynamic parameters, and mismatches in timing or duplication are reported.

Test Execution Workflow

Automated Task Trigger

The build platform supplies package information to AlphaTest, which creates playback tasks automatically as part of the agile release pipeline.

Playback Cluster Scheduling

Execution flow : a task is split into sub‑tasks, dispatched to devices, the app is installed, authorized, launched via scheme, and results are reported.

Node retry : each node retries up to three times on failure.

Sub‑task retry : each sub‑task retries up to three times for device‑level errors.

Parent‑task retry : failed sub‑tasks are reassigned to other healthy devices.

Assertion Service

During recording, assertions (text or image) are captured automatically. Playback extracts the same assertions and validates them. Image comparison uses a Siamese‑network model with a ResNeXt‑50 backbone and ContrastiveLoss, achieving >99 % similarity accuracy.

Message Push

Test results are pushed in real time through Meituan’s internal message queue and OA SDK. Templates can be customized per team.

Deployment and Impact

AlphaTest now supports more than 15 versions of the Waimai C‑end, covering ~70 % of regression cases across Native, Mach, React Native, Mini‑Program and H5 stacks. It provides UI automation, event‑tracking automation, dynamic‑load success‑rate testing and accessibility testing.

References

https://appium.io

http://docs.seleniumhq.org/projects/webdriver

http://airtest.netease.com/index.html

https://github.com/alipay/SoloPi