How to Enable White‑Box Testing on Mobile Apps with AOP‑Based Flow Recording & Replay

Background

The mobile Internet has matured over a decade, and quality‑assurance practices now focus on two directions: moving testing capabilities rightward (runtime monitoring, crash alerts, gray releases) and leftward (offline processes such as UI automation, monkey testing, performance suites). Despite this maturity, a generic code‑testing framework for mobile apps is still missing.

Code Testing Challenges

Modern mobile apps contain massive UI‑heavy codebases where UI rendering and business logic are tightly coupled, making white‑box testing difficult. Changes in SDKs or component layers often require black‑box validation because internal logic cannot be exercised directly. High execution cost and technical barriers have kept white‑box testing largely absent from most companies.

Technical Solution for Code Testing

To lower the cost and barrier of mobile white‑box testing, the article proposes an AOP‑based approach that injects pre‑written code suites into the target application. These suites can capture runtime information, inject performance monitors, or insert fault‑injection code, enabling specialized tests such as function‑level state verification.

Technical Goals

Design “flow recording” and “flow replay” code suites that, via AOP, inject themselves into the app, intercept function state, modify or store it, and combine with assertion capabilities to continuously control business‑logic correctness and testability.

Code Testability Metrics

A quantitative model is suggested that considers instance‑construction difficulty, parameter‑construction difficulty, function accessibility, and global‑variable count. These fine‑grained dimensions produce a measurable testability index, which is then complemented by qualitative analysis of common mobile code architectures.

AOP Solution

Android Bytecode Enhancement

Android AOP can be applied at source, class, dex, or runtime stages. Static injection (source/class/dex) offers flexibility but requires repackaging; runtime injection is limited because already‑loaded classes cannot be modified. Dynamic proxies, reflection, or Xposed can change logic at runtime but have usage constraints.

iOS Clang Plugin

iOS supports AOP at source, compile, link, or runtime stages. Static injection via LLVM/Clang plugins (e.g., LLVM Pass or Clang AST plugin) is preferred for controllability and flexibility. Dynamic techniques such as method swizzling or Frida are powerful but mainly used for reverse engineering, not systematic testing.

"Flow" Recording

The core of flow recording is serializing function parameters and return values. Mobile apps face additional difficulties: UI‑heavy code intertwines business logic with layout, and parameters often include complex objects (e.g., Android Context, Activity, View, Bundle; iOS UIApplication, UIView, UISceneSession). Simple JSON serialization (via fastjson, Gson, or similar iOS libraries) is recommended, with custom handling for large or circular structures.

"Flow" Replay

Replay validates recorded data by either re‑executing the original function with baseline inputs and comparing outputs, or by applying static verification rules. The main difficulty is correctly instantiating complex objects and reconstructing parameter hierarchies without losing essential information.

Solution Summary

Implementing mobile flow recording/replay requires a stable, generic AOP framework, robust serialization/deserialization handling, and a configuration‑driven object‑creation mechanism. Each stage presents pitfalls that must be addressed to maintain confidence in test results.

Overcoming Difficulties

8.1 Code Instrumentation

Android instrumentation can be built with a Gradle plugin + ASM for bytecode weaving. iOS lacks a comparable build‑time plugin system, so Clang plugins are used. LLVM’s three‑phase compiler (frontend, optimizer, backend) enables insertion of instrumentation code either during IR optimization or preprocessing.

8.2 LLVM Compilation Flow

LLVM modules (Clang frontend, optimizer passes, backend) allow custom passes to insert tracing calls. However, converting source to IR may lose parameter names/types, which can affect instrumentation fidelity.

8.3 Clang Plugin Development

The LLVM project (https://github.com/llvm/llvm-project) provides examples. A simple plugin that prints function names is shown below.

using namespace clang;</code><code>namespace {</code><code>class PrintFunctionsConsumer : public ASTConsumer {</code><code>  bool HandleTopLevelDecl(DeclGroupRef DG) override {</code><code>    for (auto i = DG.begin(), e = DG.end(); i != e; ++i) {</code><code>      const Decl *D = *i;</code><code>      if (const NamedDecl *ND = dyn_cast<NamedDecl>(D))</code><code>        llvm::errs() << "top-level-decl: \"" << ND->getNameAsString() << "\"
";</code><code>    }</code><code>    return true;</code><code>  }</code><code>};</code><code>class PrintFunctionNamesAction : public PluginASTAction {};</code><code>} // namespace</code><code>static FrontendPluginRegistry::Add<PrintFunctionNamesAction> X("print-fns", "print function names");

After building the plugin as a dylib, it can be loaded with:

clang -Xclang -load -Xclang libBiliInstrument.dylib -Xclang -plugin -Xclang bili-instrument -c DemoClass.m

Sample source before instrumentation:

#import <Foundation/Foundation.h></code><code>typedef NSString* (^CompletionBlock)(NSString *);</code><code>@interface Demo : NSObject</code><code>- (void)performActionWithCompletion:(CompletionBlock)completionBlock;</code><code>@end</code><code>@implementation Demo</code><code>- (void)performActionWithCompletion:(CompletionBlock)completionBlock {</code><code>  NSString* blockResult = completionBlock(@"bilibili");</code><code>  NSLog(@"Action Performed, %@ 
", blockResult);</code><code>}</code><code>@end</code><code>int main() {</code><code>  Demo *sampleClass = [[Demo alloc] init];</code><code>  [sampleClass performActionWithCompletion:^(NSString *name) {</code><code>    NSLog(@"Completion is called to intimate action is performed, %@", name);</code><code>    return @"block result from main";</code><code>  }];</code><code>  return 0;</code><code>}

After instrumentation, tracing calls are injected around each method.

#import <Foundation/Foundation.h></code><code>typedef NSString * (^CompletionBlock)(NSString *);</code><code>@interface Demo : NSObject</code><code>- (void)performActionWithCompletion:(CompletionBlock)completionBlock;</code><code>@end</code><code>@implementation Demo</code><code>- (void)performActionWithCompletion:(CompletionBlock)completionBlock {</code><code>  MethodTraceBegin("simple.m", "performActionWithCompletion:", completionBlock);</code><code>  NSString *blockResult = completionBlock(@"Neo");</code><code>  NSLog(@"Action Performed, %@ 
", blockResult);</code><code>  MethodTraceEnd("simple.m", "performActionWithCompletion:");</code><code>}</code><code>@end</code><code>int main() {</code><code>  MethodTraceBegin("simple.m", "main");</code><code>  Demo *sampleClass = [[Demo alloc] init];</code><code>  [sampleClass performActionWithCompletion:^(NSString *name) {</code><code>    MethodTraceBegin("simple.m", "main", name);</code><code>    NSLog(@"Completion is called to intimate action is performed, %@", name);</code><code>    int tempVar = @"block result from main";</code><code>    MethodTraceEnd("simple.m", "main", tempVar);</code><code>    return returnVar10086;</code><code>  }];</code><code>  int tempVar = 0;</code><code>  MethodTraceEnd("simple.m", "main", tempVar);</code><code>  return tempVar;</code><code>}

Engineering Practice

Both Android (Java) and iOS (Objective‑C/C++) AOP implementations are near completion, with plans to extend support to Kotlin and Swift. The first version of the flow recording/replay suite captures instance objects and function states, enabling rule‑based verification and automated test case execution.

Future work includes refining the testability metric model, improving complex‑object serialization, and providing real‑time code‑coverage previews.

Reference

[1] Software Testability Metrics and its Various Types: https://www.xenonstack.com/insights/software-testability-metrics

[2] Google Testability Explorer: https://code.google.com/archive/p/testability-explorer/

[3] Android Project Architecture Deep Dive: https://developer.aliyun.com/article/850168?utm_content=g_1000317669

[4] The LLVM Compiler Infrastructure: https://llvm.org/

[5] Clang 15.0.0 Documentation: https://clang.llvm.org/docs/ClangPlugins.html

[6] Clang Compiler User’s Manual: https://clang.llvm.org/docs/UsersManual.html

[7] The Java® Virtual Machine Specification: https://docs.oracle.com/javase/specs/jvms/se9/html/index.html

[8] See No Eval: Runtime Dynamic Code Execution in Objective‑C: https://blog.chichou.me/2021/01/16/see-no-eval-runtime-code-execution-objc/

[9] Using custom functions with NSExpression: https://funwithobjc.tumblr.com/post/2922267976/using-custom-functions-with-nsexpression

[10] Frida, dynamic instrumentation toolkit: https://frida.re/docs/home/