Image Recognition–Based Method for Measuring Mobile Web First‑Screen Load Time Using STF and Sikuli

Testing the first‑screen load time of mobile browsers has long been a challenging problem for the industry; this article introduces an image‑recognition‑based method that combines the open‑source tools STF and Sikuli to address it efficiently.

When users visit a website on a mobile device, the entire page may consist of multiple screens, but the time required to fill the visible screen (the first screen) is a critical factor in perceived performance.

The first‑screen load time is defined as the interval from the moment a user initiates a request to the moment the browser’s full‑screen area is populated with page content, allowing the user to interact without waiting for the whole page to finish loading.

Traditional measurement techniques rely on high‑speed cameras to capture the loading process frame by frame and then manually compare start and end images, which introduces timing inaccuracies due to external factors and makes the process labor‑intensive.

To overcome these drawbacks, the article proposes a low‑consumption, high‑efficiency, and highly accurate method, illustrated in the following diagram:

Steps details:

(1) Use STF to connect the mobile device to a computer, streaming the device screen to the PC with an official latency of about 30 ms; image recognition is performed on the computer using the Sikuli automation framework.

(2) Operate the mobile browser from the PC, selecting a specific image as a marker on the first screen for each test package; write Sikuli scripts for image recognition and Python scripts to collect performance metrics during the loading process.

(3) Run both programs simultaneously; when the designated marker image appears, the first‑screen load is considered complete, the timestamp is recorded, and both the image‑recognition and performance‑monitoring programs are stopped.

(4) Check whether there are remaining test packages; if so, return to step (2), otherwise proceed.

(5) Retrieve the collected data from the database and generate the final test report.

This brief introduction outlines the workflow; readers interested in deeper details about STF and Sikuli are encouraged to follow the upcoming article on their usage.