Automated Screenshot Testing for Android Fragments Using MvRx and Happo

This article is the second part of Airbnb's Android automated testing series. It explains how to use screenshot testing to automate UI verification for Fragments.

In the first article, the MvRx state‑simulation system and a launcher that can open any screen or simulate a state were introduced. While useful for manual testing, the system shines in automated testing. To leverage it, an automated screenshot test was built to detect UI changes between commits.

Benefits of automated screenshot testing include detecting:

Changes in spacing, colors, and styles

Logical issues in data presentation

Edge cases such as null or empty data

Right‑to‑left (RTL) layout problems

Changes caused by updates to common component libraries (e.g., RecyclerView, ConstraintLayout)

Manual testing of these issues is time‑consuming and often missed in engineer self‑tests and QA processes, especially for complex screens. Automated screenshot testing not only catches visual regressions but also performs basic sanity checks to ensure the Fragment does not crash.

The implementation follows three steps:

Build an Android library that captures a screenshot of an Activity and uploads the Bitmap to the cloud.

Use Happo to provide a web UI that shows bitmap differences between branches.

Configure CI to capture screenshots for each simulated state, generate a Happo diff report, and attach the result to the PR.

Creating the screenshot library involved:

Capturing the entire View hierarchy, not just the visible portion, by measuring and laying out the Activity’s root view, invoking layout and pre‑draw listeners, and drawing the hierarchy onto a Canvas.

Reducing visual flakiness by disabling EditText cursors, clearing focus from Views, forcing invalidate and requestLayout on each View, and clearing Drawable caches.

Each Bitmap is uploaded to the cloud, and the URLs are compiled into a Happo report linked to a specific Git SHA. The report can be compared with any other SHA to spot UI differences.

Performance considerations include using coroutines for concurrent processing and uploading, handling potential OutOfMemory errors, truncating simulated list lengths to three items while still supporting content beyond one screen, reusing Bitmaps efficiently, and employing exponential back‑off retry logic for unstable network requests to AWS and Happo.

Happo is an external service for bitmap comparison that offers features such as historical bitmap views, email alerts on changes, flaky‑diff marking, PR blocking until diffs are approved, and a web UI for reviewing all screenshots.

CI integration ties everything together: each PR runs a blocking workflow that generates screenshots, compares them via Happo, and posts a comment with the diff and a link to the report.

Additional tests integrated into the framework include:

LeakCanary checks to detect Fragment, View, or Activity leaks after each screenshot.

Espresso accessibility checks on the rendered Activity.

Simulation of process death and recreation to verify that Fragment arguments and state survive without crashes, with post‑recreation screenshots for verification.

The framework abstracts away the need for developers to write extra code for screenshot testing; they only need to define simulated states in their Fragment classes.

The series will continue with parts covering interaction testing, ViewModel unit‑test frameworks, architecture of Airbnb’s automated testing framework, consistency of simulations, and test code generation with CI configuration.

Series overview: the series consists of seven articles covering principles, state simulation, automated screenshot testing (this article), automated interaction testing, ViewModel unit‑test framework, architecture, simulation consistency, and test code generation.