Ensuring Consistency in Airbnb's Android Automated Testing Framework

In the sixth article of the Airbnb automated testing framework series, the authors discuss common factors that cause inconsistent test results and describe how they address these issues.

The main challenges include nondeterministic test ordering caused by Flank’s dynamic sharding, asynchronous code execution (RxJava, Kotlin coroutines, AsyncTask, Executor, custom threads), view caching (invalidate(), requestLayout()), shared preferences leakage across simulated states, fluctuating system time/date, drawable caching, memory‑related crashes, delayed runnables, non‑simulated states, image loading, WebView loading, and RecyclerView pre‑fetching.

To mitigate these problems, the team implements several strategies:

Manually clear shared state between test fragments and prevent memory leaks by limiting test shard duration and using LeakCanary.

Provide a custom async‑runner function that can block or detect the completion of asynchronous code, and inject a test‑scoped CoroutineScope into ViewModels.

Force view hierarchy refresh by calling invalidate() and requestLayout() on all views.

Clear SharedPreferences (and other storage) after each simulated state.

Mock time and date using a wrapped JodaTime API so that now() and today() return constant values.

Clear drawable caches after each screenshot by triggering a configuration change with the following code:

AppCompatDrawableManager.get().onConfigurationChanged(view.context)
val resources = view.resources
val currentConfig = resources.configuration
val currentDisplayMetrics = resources.displayMetrics

resources.updateConfiguration(null, currentDisplayMetrics)
resources.updateConfiguration(currentConfig, currentDisplayMetrics)

Enable android:largeHeap and reuse a single bitmap for large screenshots to avoid OOM.

Disable RecyclerView pre‑fetching by calling setItemPrefetchEnabled(false) on all LayoutManagers after the fragment is laid out.

Replace image loading with synchronous local drawables during tests and capture the intended URL in JSON reports.

Wrap WebView in a custom view to block network loading and record URL, user‑agent, and headers in test reports.

The article concludes by summarizing these consistency‑preserving techniques and previewing the next piece, which will cover test code generation and continuous integration.