Intelligent Component Testing Solution for Youku Mobile App

Background With the increasing richness of content operation schemes on the Youku APP, the number of operation components on the content configuration platform has grown, leading to a surge in regression testing workload on the mobile side. This article shares Youku's thoughts and explorations on maintaining high‑efficiency component testing quality while keeping pace with business development.

Current Situation Analysis

Many components (hundreds) result in high regression cost; rapid coverage testing is needed during version regression.

Metadata diversity and personalized operation strategies cause massive adaptation work for UI upgrades or technical refactoring.

Existing UI automation only verifies component view existence, not detailed controls (e.g., titles), and requires separate native and cross‑platform locating scripts.

Automation based on native locating is fragile to technical refactors, increasing maintenance cost.

By analyzing these issues, Youku combined existing Mock and image‑recognition capabilities to build a component‑centric intelligent testing solution, aiming to test mobile components as if they were server‑side APIs.

Project Solution

The solution addresses the problems from five aspects (illustrated in the original diagram).

Solution Value

Leverage Mock to create a data‑driven component testing approach, improving testing efficiency and user experience.

Use image recognition and Mock to reduce dependence on the client testing system and enhance test stability.

Apply server‑side traffic data to the client, building a data factory that expands test data coverage.

Overall Architecture

Data Factory

The data factory abstracts and stores online traffic data and tool‑generated data, supporting single‑slot and multi‑slot templates, normal and abnormal data generation, and rapid construction of rich component test data. Phase 1 is completed, providing platformized capabilities such as Mock data editing, combination, generation, and configuration file creation.

Image Recognition Capability

Youku’s image‑recognition service enhances component verification and reduces verification difficulty. The solution evolved in two phases:

Phase 1 : Deep‑learning algorithms extract component layout rules, mapping page layouts to server‑side data structures. This phase has high learning cost and limited flexibility for business extensions.

Phase 2 : A component annotation interface allows business users to upload screenshots, annotate layout layers, and obtain annotated IDs. Automation then sends the screenshot and annotation ID to the service, which returns recognized text and coordinates.

Automation Testing

With data from the data factory and layout‑recognition capability, a data‑driven intelligent component testing framework is built on top of Youku’s mobile automation framework (derived from Ant Financial’s mobile automation framework). Business users only need to create test‑data configuration files and implement business logic; the common case handles UI comparison and click verification.

Basic Flow and Logic

The automation combines similarity‑based image comparison with configuration‑driven layout recognition, performing UI verification and click verification in two steps. Depending on the presence of baseline images, annotation IDs, and similarity results, the test follows one of five logical paths (detailed in the original flow diagram).

Project Impact and Application

Automation has been integrated into the homepage and channel pages, covering over 90% of frequently used components and automating more than 60% of component test cases. Screenshots demonstrate similarity‑based comparison plus configuration‑driven coordinate clicks, as well as layout UI comparison for multiple slots.

Future Plans

Automation side : Optimize common capabilities, extend to other test types (e.g., adaptation testing) and to Pad devices; integrate test triggering into the data platform for one‑click execution.

Image recognition side : Strengthen the Phase 2 annotation interface with visual editing and multi‑slot component labeling.

Data factory side : Refine abnormal data construction rules, cover more business strategies, generate comprehensive business data cases, and leverage online data to replay hot‑path scenarios.