Precise App Response Time Measurement Using Frame-Based Analysis

Startup time is the most direct performance experience for an app (especially cold start); users often have to wait several seconds before the app becomes usable, which is intolerable. The response time of various operations also directly impacts user experience, so objective testing of response time—including both cold and hot start times—is needed to drive targeted product optimizations.

The principle of response‑time testing is to measure the interval from the moment a user taps the screen (the point where the program receives the command) to the moment the expected UI is fully displayed. A shorter interval indicates faster response and stronger capability of the app to react to user commands.

Two conditions are required for testing this interval: (1) precisely locate the start point (when the app receives the user command) and the end point (when the UI is fully rendered); (2) accurately measure the time between these two points, which merely needs a millisecond‑precision timer.

Four common methods are described:

Method 1: embed logging points in one or two related activities and calculate the time difference between the logs.

Method 2: use image‑recognition by taking screenshots of the whole page or a specific region and comparing them to detect when the target appears.

Method 3: employ Android automation to detect specific UI elements and calculate the time when the element appears.

Method 4: record the process with a high‑speed camera, then analyze the video frame‑by‑frame to determine the start and end frames.

Each method has drawbacks: logging depends on developers placing points accurately; image‑recognition adds runtime overhead and may fail on dynamic pages; automation requires extensive test case maintenance; high‑speed camera needs extra PC software, is complex to set up, and has higher cost.

Based on the analysis, a frame‑analysis tool was developed to provide an easy‑to‑use and extensible platform for precise app response‑time calculation.

Tool principle : Users upload a video (preferably MP4) captured via screen recording or a high‑speed camera. The backend uses ffmpeg to extract every frame, then applies MD5 deduplication to keep only the first and last frames of identical static pages. Afterwards, pyssim removes similar frames with a similarity threshold of 90 %, preserving the first and last frames of each similar block. This reduces the number of images while retaining key frames needed for timing.

The processed frames are displayed on the platform; users select the start frame and end frame for each scenario (e.g., app launch, button click) and the tool computes the response time, storing results for reporting and comparison across versions or competitors.

Tool usage : Platform URL – http://metric.baidu.com:8080/duapps/restimeupload; PDR toolset – http://pdr.baidu.com/ (download Pandora Android tools). After uploading a video, the system processes it and notifies the user upon completion. Users then select one or multiple scenes, choose the appropriate start and end frames, click “Calculate,” and obtain the response‑time results, which are saved to the database.

Advantages :

Frame‑based calculation yields high accuracy with minimal error.

Avoids image‑recognition errors and issues with dynamic pages.

No additional cost; works across iOS, Android, and competitor apps.

No extra development effort; applicable to various scenarios.

Can also be used for WAP or WebView applications.

Disadvantages :

Processing time is relatively long, but the user can continue other work while waiting for the notification.

Videos with varying frame rates may introduce slight errors, though typical screen recordings are stable.

Automation is not yet integrated; future work could allow automatic baseline‑image comparison, which would re‑introduce image‑comparison challenges.

Final timing depends on the user’s manual selection of start and end frames, though the high similarity threshold reduces the chance of picking wrong frames.

Author : Wu Jian, Baidu – EBG Testing Department, with experience in recommendation, personalization, and internationalization testing, enjoys building small tools that save time and effort.