Capturing and Replaying Mobile App UI Issues with Unified Hooking and Touch Simulation

Background

After an app is released, developers often struggle with user‑reported problems because users cannot provide reproducible steps, and developers cannot easily locate the issue. The article proposes a new technical approach to capture UI event flows and related data, then replay them to reproduce problems.

Benefits of an Online Issue Replay System

Users can reproduce the problem by simply re‑executing the recorded steps without typing text.

Developers receive a script that replays the exact UI actions, similar to watching a video.

The replay also captures runtime data (local, network, stack) for deeper analysis.

The framework can be reused for automated testing.

Overall Architecture

The system consists of two parts: recording and replay. Recording captures UI events and data; replay drives the app to reproduce the same state.

Key Technologies

1. Simulating Touch Events

The core idea is to extract the touched view and its hierarchy from the UI event stream, locate the corresponding view in the current UI hierarchy, and inject a synthesized UITouch event with proper coordinates, timestamps, and phase.

// Simplified touch simulation flow
- Capture UI event stream
- Record UIEventTypeTouches with timestamp, view, location
- On replay, create a UITouch object
- Set its phase (Began, Moved, Ended) and location
- Wrap it in a UIEvent and send via [[UIApplication sharedApplication] sendEvent:event];

2. Unified Interceptor (Hook)

All methods that need to be recorded or replayed are hooked by a low‑level assembly interceptor. The interceptor collects register values, the self object, and the selector, then looks up a corresponding ReplayEventIns node from EngineManager. The node provides callbacks to decide whether to replay, fetch replay data, and inject it back into registers or object fields before optionally calling the original method.

void replay_entry_start(void *xRegs, void *dRegs, void *spReg, CallBackRetIns *retIns, StackFrame *fp, void *con_stub_lp) {
    void *objAdr = ((void **)xRegs)[0];
    ReplayEventIns *node = [manager getEventInsWithBlock:objAdr];
    // Resolve selector, class hierarchy, find node
    // Call willReplay / getReplayData callbacks
    // Inject data via sstuffArg / fetchAllArgInReplay
    // Set retIns fields for continuation or early return
}

3. Unified Hooking of Objective‑C Blocks

Blocks are represented by internal structs ( block_info_1, block_descriptor_head, etc.). The solution creates a new wrapper block whose invoke pointer is replaced with a unified callback ( hook_var_block_callBack_record or hook_var_block_callBack_replay). The original block and its associated ReplayEventIns are stored in the wrapper’s captured variables, allowing the callback to retrieve the original context and perform recording or replay.

VoidfunBlock createNewBlock(VoidfunBlock orgBlock, ReplayEventIns *blockEvent, bool isRecord) {
    if (orgBlock && blockEvent) {
        VoidfunBlock newBlock = ^{ orgBlock(); };
        trace_block_layout *layout = (__bridge trace_block_layout *)newBlock;
        layout->invoke = isRecord ? hook_var_block_callBack_record : hook_var_block_callBack_replay;
        return newBlock;
    }
    return nil;
}

Implementation Details

The article also provides low‑level C/assembly snippets for extracting registers, building IOHIDEvent objects for touch simulation, and handling block descriptors. It explains the block memory layout (stack, heap, global) and the flags that describe copy/dispose behavior.

Conclusion

The presented framework enables developers to capture user‑side UI problems, store them as replay scripts, and automatically reproduce them, reducing debugging effort and improving reliability of mobile applications.