iOS Stability Testing: Framework, Strategies, and Optimization Practices

Stability testing is the practice of running an app continuously for a long period to verify its robustness. It helps uncover intermittent crashes, memory leaks, and performance degradation that may only appear after prolonged usage.

The iQIYI testing team has explored iOS stability testing extensively and shares their practical experience, including the overall system architecture, event generation strategies, and handling of runtime interferences.

System Framework

The iQIYI iOS stability testing platform builds on an existing cloud device farm and consists of three main layers: device management at the bottom, result aggregation at the top, and the core testing strategy in the middle. Real devices are accessed through a driver layer, centrally scheduled by the backend. Test events are generated, executed on the devices, and the resulting data is collected and reported.

Figure 1: System framework

Event Generation Strategies

Two main strategies are compared:

Random event generation (Monkey testing) – low implementation cost, but many generated events are ineffective (up to 90% invalid) when the UI has few clickable elements.

Element‑based traversal – higher implementation cost but yields more effective coverage by clicking identified UI elements.

Random events are implemented using XCTest APIs to simulate clicks, swipes, home button presses, etc. The configuration defines the probability of each event type, and the driver service executes the assembled event sequence.

Figure 2: Random event generator

For element‑based generation, three methods of element identification are evaluated:

Integrating a custom SDK into the app – high integration cost and often prohibited in release builds.

Parsing the DOM tree – suffers from accumulated elements and difficulty distinguishing the current page.

AI image‑based element recognition – leverages an internal AI service to analyze screenshots and locate UI elements accurately.

Figure 3: AI service recognizing page elements

To avoid revisiting the same page during traversal, a fingerprinting technique based on screenshot hashing is used. Screenshots are converted to 8×9 grayscale thumbnails, hashed, and compared with a similarity threshold to determine whether two pages are identical.

Figure 4: Screenshot‑based fingerprint for page similarity

Handling Runtime Interferences

During long‑running tests, unexpected pop‑ups, app backgrounding, or stuck pages can block further actions. The framework implements several mitigation techniques:

Detecting and clicking back/close buttons by matching DOM attributes containing keywords like “back” or “close”.

Using OCR to recognize the word “返回” (Back) on the screen and clicking it.

Training an AI model to locate “X” close icons on ad overlays.

Pop‑up detection is optimized by using NSPredicate regular expressions, allowing a single request to check for multiple possible texts, reducing network overhead.

Figure 7: Example pop‑up handling with combined predicates

Ensuring the tested app stays in the foreground is achieved either by inspecting the DOM for the app’s bundle identifier or by calling the Apple‑provided activeApplication API to retrieve the current foreground bundle ID.

Evaluation Results

Each strategy was executed 20 times for 8 hours, measuring total events, invalid‑event ratio, step latency, and page‑coverage. The element‑traversal strategy achieved roughly 30 % higher page coverage than random clicks, though its per‑step latency was higher due to image processing.

Figure 10: Strategy comparison table

Figure 11: Page coverage over time

In production, the stability testing service runs over 800 times per month, detecting more than 200 crashes and serving as a critical component of the DevOps pipeline.

Future work includes more targeted traversal strategies, such as weighting pages based on real‑user traffic or historical crash frequency.