How Fastbot Boosts iOS App Stability with AI‑Driven Automated Testing

Background

After Apple limited TestFlight, a single app’s gray‑scale user count is restricted, making it hard to expose stability issues offline. Fastbot, jointly developed by ByteDance Quality Lab and the GIP iOS platform architecture team, uses machine learning and reinforcement learning to provide automated stability testing that outperforms traditional Monkey testing, doubling problem detection and code coverage.

Integration History

Since the second half of 2020, the GIP iOS platform team has worked on Fastbot, focusing on three areas: generic capability development (visual operations, freeze‑screen detection), capability extension (custom business scenarios, UI checks, keyboard optimization), and data consumption (offline release process, test‑device allocation strategies).

Based on these practices, a Fastbot‑driven automated testing workflow was built, forming a closed loop from issue discovery to consumption, and enabling precise targeted testing for each MR by constructing a code model from the repository and selecting affected view controllers.

Fastbot Problem Discovery Enhancements

1. Scene Limiting

To improve traversal and issue detection for deep view controllers, Fastbot implements a scene‑limiting feature that keeps testing within a specific VC reachable via a scheme, using a WebServer in a dynamic library to hook UIViewController presentation and push methods, monitor entry/exit, and repeatedly invoke the target scheme when the app leaves the limited scene.

The feature has increased abnormal interception rates from under 5% to over 20% in various activity scenarios.

2. Accessibility Testing

With over 17 million visually impaired users in China, Fastbot runs automated accessibility checks using Google’s GTXLib rules during traversal, reporting violations such as punctuation in accessibility labels, touch‑target size, label‑trait overlap, missing labels, dynamic type support, contrast ratio, and accessibility traits.

UIAccessibilityTraits UIAccessibilityTraitNone; // No traits
UIAccessibilityTraits UIAccessibilityTraitButton; // Button
UIAccessibilityTraits UIAccessibilityTraitLink; // Link
UIAccessibilityTraits UIAccessibilityTraitSearchField; // Search field
UIAccessibilityTraits UIAccessibilityTraitImage; // Image
UIAccessibilityTraits UIAccessibilityTraitSelected; // Selected
UIAccessibilityTraits UIAccessibilityTraitPlaysSound; // Plays sound
UIAccessibilityTraits UIAccessibilityTraitKeyboardKey; // Keyboard key
UIAccessibilityTraits UIAccessibilityTraitStaticText; // Static text
UIAccessibilityTraits UIAccessibilityTraitSummaryElement; // Summary element
UIAccessibilityTraits UIAccessibilityTraitNotEnabled; // Not enabled
UIAccessibilityTraits UIAccessibilityTraitUpdatesFrequently; // Updates frequently
UIAccessibilityTraits UIAccessibilityTraitStartsMediaSession; // Starts media session
UIAccessibilityTraits UIAccessibilityTraitAdjustable; // Adjustable
UIAccessibilityTraits UIAccessibilityTraitAllowsDirectInteraction; // Direct interaction
UIAccessibilityTraits UIAccessibilityTraitCausesPageTurn; // Causes page turn
UIAccessibilityTraits UIAccessibilityTraitHeader; // Header

Fastbot logs each touchBegin event, aggregates violations into GTXResult structures, and reports them to the server.

3. Task Exception Recovery

3.1 Freeze‑Screen Detection

Fastbot detects when the app is stuck by computing a sid (MD5‑like hash of the UI tree) for each operation; if the most frequent sid occupies over 87.5% of the last 30 samples, a freeze is assumed and Fastbot triggers a recovery flow (force back, handle system and app dialogs, force restart).

This reduces test‑device waste by about 10% and logs the frozen pages for later analysis.

3.2 Keyboard Scene Optimization

Fastbot clusters keyboard keys into a single control based on identifiers like “KBKey” or “UIKeyboardEmoji”, limiting keyboard activation to five times per lifecycle and capping each session at three minutes; excess keyboards are force‑closed, and tap gestures are simulated to exit.

After optimization, keyboard time share dropped from 20% to under 5% in a 10‑hour test.

Fastbot Result Consumption

Metrics such as recall rate, coverage, offline crash rate, and device‑hour completion are consolidated into a three‑part offline release workflow: quality assessment, exception consumption, and release gate, providing a unified view of app quality and guiding release decisions.

The workflow has supported ten GIP releases, saving roughly two person‑days per version.

Impact

Daily automated builds now achieve around 80% VC coverage, with issue recall improving from 19.1% to 45.76% and targeted MR testing execution reaching 53.03%.

Future Outlook

Fastbot will continue to enhance traversal capabilities, enrich metrics, integrate offline recording for replay, and expand UI assertions and performance testing to further reduce manual testing effort.