Meituan Technical Salon #77: Client‑Side Robustness Testing via Interface Data Mutation for Billion‑Traffic Systems

In the Meituan Technical Salon #77 talk, robustness is defined as a system’s ability to handle errors and continue normal operation despite abnormal inputs or environmental stress, drawing on definitions from Wikipedia, IEEE, early fuzz testing (Barton Miller, 1989) and the Ballista project (1996). For mobile apps, robustness means the client can keep running when faced with environmental or input anomalies such as memory shortage, high CPU load, network failure, or malformed data returned from APIs.

Analysis of historical faults in Meituan’s platform showed that the majority of crashes and core‑function failures stem from unexpected network‑response data, making client‑side robustness testing a priority. The proposed solution centers on mutating the original API response according to semantic rules, injecting the mutated data via a proxy, and detecting crashes, hangs, or functional abnormalities on the device.

The detection scheme combines static and dynamic components. Static detection uses custom code‑scanning rules (e.g., PrimitiveParseDetector, ColorParseDetector) to enforce safe usage of utility classes and manage technical debt. Dynamic detection constructs mutated data, injects it at appropriate trigger points (integration/regression events), and monitors the app for abnormal behavior.

A key challenge is generating comprehensive yet manageable mutation data. The team performed semantic analysis of response values—distinguishing colors, URLs, image resources, etc.—and defined initial mutation rules based on boundary values and observed production issues, later enriched with real‑world fault data.

To curb the exponential growth of test cases, two optimization strategies were applied: (1) when array elements share identical structure, assign distinct mutation values to each element and discard redundant combinations; (2) for structurally varied arrays, compute edit distance between nodes, convert it to a 0/1 similarity matrix, apply union‑find to group similar nodes, and pick one representative per group for mutation. These tactics reduced the generated test suite by roughly 40% without losing coverage or defect‑detection ability.

The resulting tool has been integrated into both Meituan and Youxuan apps. It can be triggered manually for new features, automatically upon component changes, or as part of bi‑weekly regression runs. Over one year of use, the approach has uncovered dozens of robustness issues.

Future work includes enriching the mutation model with more data and environment types, improving detection capabilities, and exploring smarter, AI‑driven test‑case generation to further boost efficiency.