Xianyu Mobile Performance Testing Automation Platform Solution

This article introduces the solution and implementation process of Xianyu's mobile performance testing automation platform. Initially, Xianyu's performance testing process had low automation coverage, requiring manual intervention in almost all steps, which was time-consuming and labor-intensive. The main problems included high manual costs, large manual operation errors, lack of performance checkpoints during testing, no data accumulation, and difficulty in self-testing by R&D personnel.

To address these issues, Xianyu designed and developed a mobile performance testing platform to improve testing efficiency through process standardization, automation, and data accumulation. The system design follows minimalism principles, directly using existing SLM and TMQ capabilities for performance data collection, and using DeepInsight to generate reports. The platform integrates with the entire packaging and building process, automatically triggering performance tasks when release packages are successfully built.

The specific testing process is divided into three stages: task triggering, automated execution, and PTM/PM data confirmation and problem investigation. The platform includes an automatic retry mechanism to verify data accuracy by comparing with historical versions. During implementation, several challenges were encountered and solved, such as the impact of machine temperature on performance, network loading affecting FPS, and device differences affecting test results.

After a period of practice, the platform has been applied to Xianyu's release process (versions 6890-6940), achieving significant results: reducing manpower costs from 1 person-day to within 1 hour, improving data stability and accuracy, and accumulating valuable performance data for deeper analysis. Future plans include further left-shifting testing, integrating with continuous integration for testing checkpoints, and deepening data analysis capabilities to provide more precise problem localization.