Automated Regression Express System (ARES): A Module-Level Automated Testing Framework

Abstract

ARES (Automated Regression Express System) is a module‑level automated testing tool that records real online data and replays it in an offline environment, treating the tested module as a black box; it offers low integration cost, simple deployment, fully automated testing, and high coverage.

Background

Complex systems face testing challenges: defining test scope becomes difficult as business logic evolves, and case verification incurs high cost due to data and environment preparation. Traditional online diff and manual diff approaches have limitations, prompting a shift toward isolating invariant factors and focusing unit tests on changed modules.

Unit‑Testing Challenges

Traditional unit testing requires maintaining test code, mocks, and assertions, which can become burdensome and quickly outdated as business logic iterates, leading to low effectiveness.

Implementation Principle

ARES operates in two phases: dump (online data collection) and replay (offline data playback). The tested module is treated as a black box; its inputs, outputs, and runtime environment are captured online, then mocked offline for replay.

Overall Process

1. Dump Process : ares‑client embedded in the module collects online data and pushes it to mock‑server. 2. Replay Process : ares‑server deploys the module in a Docker container, retrieves dump data from mock‑server, optionally runs coverage analysis, restores requests, and replays them using ares‑client; results are pushed back to mock‑server, compared by diff‑server, and reported.

ARES Modules

ares‑client : embedded in the tested module; consists of ares‑client‑base (core API & SPI), ares‑client‑common (generic hooks), and ares‑client‑target (platform‑specific hooks). It is business‑agnostic and supports various interaction types (HTTP, RPC, MQ) via plugins.

ares‑server : core server handling task creation, case management, environment deployment, and result visualization.

mock‑server : stores online request/response data, runtime configuration, downstream calls, and replay results, enabling replay, mock, and diff operations and future extensions such as case search and editing.

diff‑server : compares dump data with replay results, focusing on return values, downstream call parameters, and call order to determine test pass/fail.

How to Use

Example: a hotel ordering flow with modules price, book, order, product. If only the book module changes, ARES can dump data for book, replay it, and verify that book’s outputs and interactions remain unchanged, confirming safe deployment without rebuilding the entire system.

Advanced Features

Multi‑module joint testing extends the tested module concept to a set of changed modules, allowing combined replay of price and book, for instance.

Mock data tools aim to leverage stored dump data for case search, editing, and generation, facilitating targeted testing and rapid case creation for evolving business logic.

Conclusion

ARES simplifies testing of complex systems by isolating invariant factors, turning regression testing into module‑level unit testing with automated data collection and replay. It is especially suitable for business regression and internal refactoring, though less applicable to brand‑new features or highly dispersed changes. Future work includes improving client integration, API robustness, concurrent replay, and expanding advanced functionalities.