Robustness Testing of iQIYI Mobile App Using Dirty Data Injection

Crash problems caused by backend data‑format changes are a major threat to the stability of mobile apps. In iQIYI’s case, incompatibility between the front‑end and changed server responses leads to hard‑to‑detect crashes during feature iteration.

To address this, iQIYI’s technology product team designed a robustness‑testing solution that injects “dirty data” at the network layer. A custom SDK intercepts HTTP responses, replaces the original JSON with engineered test data, and feeds the mutated payload back to the app, allowing early discovery of data‑change‑induced crashes.

The solution starts with a web‑based strategy configuration console. Users define the URLs to be tested and create data‑mutation rules through a visual interface. The console supports fuzzy domain matching, multi‑domain policies, and rich data specifications (single‑node, multi‑node, removal, insertion, emojis, nulls, objects, etc.). These rules are stored as reusable strategies.

At runtime, the SDK uses an ASM‑based hook framework to modify the bytecode of the underlying network library. When a request matches a configured URL, the SDK intercepts the response, applies the configured mutation rules, and returns the altered JSON to the caller. The approach is non‑intrusive, works across different network libraries, and can be toggled on or off independently of other modules.

Two traversal strategies are provided: (1) sequential node refresh, which walks each JSON node in order and guarantees full coverage but may be slower for large payloads; (2) random full‑node traversal, which selects a random dirty‑data entry for each matching request, offering high randomness and suitability for stability testing. A local stack persists traversal state, enabling recovery after app crashes or restarts.

The platform supports three testing modes:

UI automation : scripted navigation to target pages, followed by dirty‑data injection to verify UI behavior under mutated responses.

Stability (Monkey) testing : random dirty‑data injection across many pages to achieve broad chaos testing with low cost.

Manual testing : flexible, low‑cost testing for scenarios hard to automate (e.g., QR code scanning, payment flows), with a clear execution flow: configure strategy → install APK → launch → input strategy ID → run tests → automatic defect reporting.

Since deployment, the solution has been adopted by multiple business lines, uncovering numerous hidden defects, achieving a defect‑closure rate above 50%, and significantly improving code robustness and stability. Future plans include expanding data‑node coverage, enriching dirty‑data types, automating data extraction for versioned APIs, and enabling distributed execution across multiple devices.