Automated Testing Framework for Baidu Waimai User Profiling Using Asynchronous Coroutines

In Baidu Waimai’s O2O business, offline data systems are crucial for building detailed user and merchant profiles, but the sheer number of fields (over 800) makes manual testing costly and error‑prone. The article outlines the evolution of the testing approach from a fully manual stage (200+ fields) to a partially automated stage (300+ fields) and finally to a fully automated, asynchronous coroutine framework handling more than 700 fields.

The initial 1.0 automation read field‑SQL pairs from input files, executed the SQL, and performed a simple diff against strategy output, requiring extensive manual verification. Performance bottlenecks and limited diff capabilities led to the development of version 2.0, which introduced producer‑consumer coroutine patterns, concurrent HTTP requests, and specialized diff methods for complex data types.

The 2.1 framework layers fields by business relevance, assigns different user counts per layer, and processes SQL and HTTP calls concurrently without blocking, dramatically reducing execution time. Test results are organized into six sections, including per‑user diff details, error counts, and overall execution metrics.

Beyond strategy testing, the framework was extended to automate API and Elasticsearch (ES) data verification, handling string‑based diffs for ES and accounting for data transformations in API responses. The reuse of the framework provides comprehensive coverage of the entire data pipeline.

Concluding remarks highlight the framework’s suitability for I/O‑intensive workloads, its impact on testing efficiency (automation rate up to 86%), and future plans to increase coverage for complex strategy fields, incorporate white‑box testing, and add data‑flow monitoring with alert integration.