How Scenario‑Based Automated Testing Boosted Efficiency for a Fast‑Growing Audio Platform

Background and Purpose

The audio platform experienced explosive growth, making its backend increasingly complex. The testing team faced dual pressure to improve test quality while controlling regression cost and delivery speed, prompting the need for comprehensive business‑logic coverage through automated testing.

Framework Overview

The solution, named FAST – Soldier‑Style Shell Helmet , emphasizes lightweight design, modularity, and proactive threat detection. Design principles include:

Isolation : front‑end, back‑end, and data are isolated to avoid interference.

Less Is More : simplify interaction and functionality through refactoring.

Unified Stack : React + Ant Design + Java + MySQL + Redis + Message Queue.

Scenario‑Based Testing System

Key Questions

When should regression be triggered? How to handle services that need periodic testing after release?

What is the regression scope? Which business logic and downstream services are affected?

How to collect test results?

How to prevent problematic releases?

Core Capabilities

Test resources are decoupled from each delivery

Diverse trigger mechanisms aligned with the platform’s release system (daily, gray, scheduled).

Broader coverage that discovers issues earlier, including downstream scenarios.

Cost compression

Visualized workflow.

Multiple import methods (case, curl, module).

Integrated authentication (passport, ops) for automatic login.

Custom services for ordering, signature verification, mock parameters, etc.

Data sharing and transmission.

Unattended execution.

Monitoring and feedback

DingTalk notifications for developers, testers, and owners.

Robot group alerts.

Dashboard visualizations.

Integration Strategy (Recommended)

The process has been adopted across multiple business lines and can serve as a standard workflow.

Core Concepts

Scene Set : equivalent to a project.

Test Scene : similar to a test‑case collection.

Step : equivalent to a testcase; the smallest effective unit, representing an interface call plus validation.

Execution Path

Preparation Identify which business flows can transition from manual to automated regression based on existing test cases. Discuss internally to classify services (e.g., Xdcs application grading, traffic grading) and select high‑priority (S‑level) flows.

Formal Construction Build scene sets, assign them to groups, and allocate to individuals. Define shared test data (e.g., album IDs) and non‑shared data (e.g., user accounts).

Execution Run the automated scenes, collect results, and generate reports.

Feedback Send notifications to individuals and groups via DingTalk and robot channels, and update the dashboard with the latest test status.

Core Value and Outcomes

By the second half of 2022, the scenario‑based testing framework was fully established. Multiple teams integrated it, achieving batch testing without manual intervention and meeting the first‑stage goals. The accompanying charts illustrate the operational benefits and cost savings realized over a six‑month period.

Future Direction and Trends

The team plans to further refine the framework, expand scenario coverage, and integrate more intelligent monitoring to continue enhancing testing efficiency and reliability.