How ByteDance’s SmartEye Boosts Android Testing with Precise Call‑Chain Analysis

Background

Automated precise testing aims to automatically and accurately test the code changes in each merge request (MR) to improve code quality and reduce manual testing effort.

Existing Process

Typical development workflow includes developer white‑box testing and tester black‑box testing. Tests are recorded per Activity, and for each MR the tester runs the relevant test cases.

MR changes are abstracted into three cases: adding new methods, modifying existing methods, and deleting methods. New features may require additional test cases, while modifications or deletions can be covered by existing cases.

Automated Testing Workflow

Manual execution of test cases is repetitive, so the workflow is automated by integrating test execution into the CI pipeline, using a cloud device farm to run the tests.

Challenges encountered include the high cost of running all test cases for every MR and the small size of most code changes, making full test execution inefficient.

Precise Testing Solution

The goal is to recommend test cases that are directly related to the code changes in each MR.

How to associate test cases with code?

How to obtain the changed methods in an MR?

How to accurately recommend test cases?

Linking Test Cases to Code

Test cases are recorded as user actions per Activity. By determining which Activity a piece of code belongs to, only the relevant test cases need to be executed.

Method Call Chain

A method call chain graph is built by connecting all function calls in the code. This graph, combined with Activity information, forms an Android call‑chain graph that links Activities to the methods they invoke.

Android Call‑Chain Capabilities

Query all functions involved in an Activity.

Query all Activities related to a function.

Query one‑hop, two‑hop, etc., function call relationships.

Identify the entry function of an Activity.

Find the next Activity in a navigation flow.

Obtaining MR Change Content

Instead of using raw diff, the focus is on the changed methods (added, modified, deleted). Static analysis with IntelliJ PSI is used to compare the pre‑ and post‑change source files and detect method‑level changes.

Static Analysis Rules

New method: appears only in the new file.

Deleted method: appears only in the old file.

Modified method: present in both files but with different body sizes.

SmartEye Automated Precise Testing

SmartEye builds a data pipeline that records manual and automated test cases, parses coverage data, and generates test‑case indexes based on call‑chain analysis, enabling accurate test‑case recommendation and reducing testing effort.

Fastbot Automated Precise Testing

Fastbot leverages the same MR change analysis and call‑chain mapping to trigger targeted automated tests, achieving “test what changed” functionality and improving defect detection.

Android Call‑Chain Construction Process

Stage 1: Global Call‑Graph Generation

Using the Soot framework, the APK is parsed to extract all classes, methods, and method bodies. Edges are added between methods whenever a call is detected, resulting in a complete call‑graph.

Stage 2: Activity‑Method Call‑Chain

All Activity classes are identified by checking inheritance from android.app.Activity or androidx.appcompat.app.AppCompatActivity. For each Activity, the call‑chain is generated by traversing the global graph from the Activity’s entry methods.

Call‑Chain Optimization: Linking Android Native Components

Fragments and custom Views are often loaded via layout files or dynamically, which are not captured by the basic call‑graph. To associate these components with Activities, the layout files are parsed to map resource IDs to XML definitions, and edges are added from Activities to the corresponding component methods.

Benefits

Test efficiency improved by 35% for Douyin Android projects.

Activity coverage increased ~15× compared to generic automation, discovering ~3 crashes per task.

Conclusion and Outlook

The Android call‑chain enables precise mapping from code changes to test cases, greatly enhancing testing efficiency. Future work includes handling asynchronous calls, refining the call‑chain to basic‑block granularity, and extending the approach to other analysis scenarios such as sensitive method tracing.