AREX: An Open‑Source Automated Regression Testing Platform for Real Production Traffic at Ctrip

As Ctrip’s ticket business grew, the testing team faced rising cost and complexity: high data‑construction effort, extensive regression suites, difficult test‑case reuse, and heavy manual maintenance. The core problems were cost constraints and escalating system complexity.

To address these challenges, the team explored several solutions, including an AUTO test platform, TCP replay, script‑generated test data, multiple mock services, coverage metrics, and optimized test environments. However, two persistent issues remained: automation still required manual maintenance, and large‑scale data‑construction and frequent releases generated massive workload.

The new goal was clear – improve quality without excessive human effort. The resulting idea was to perform regression testing with real production traffic and data, leading to the creation of AREX, an open‑source platform (https://github.com/arextest) that combines recording, replay, and diff comparison.

AREX consists of a Java Agent and a set of services (frontend, storage, scheduler, report, database). It records near‑full‑scale request flows and associated data in production, stores them in a dedicated database, and replays them in a test environment where third‑party calls are mocked. After execution, a comparison SDK evaluates differences and generates a test report.

The platform’s technical backbone includes:

ByteBuddy for byte‑code instrumentation, enabling injection of recording and replay logic into target methods.

SPI‑based plugin architecture for extensible module, type, and method instrumentation.

Thread‑local Record ID generation to preserve call‑chain context across multithreaded and asynchronous code.

Version‑aware injection that reads JAR manifest information to apply the correct instrumentation for different library versions.

Class‑loader isolation to protect the application from agent side‑effects.

Time‑mocking via a custom currentTimeMillis() proxy to avoid expiration issues during replay.

Dynamic cache mocking and configurable mock rules to handle stateful components.

AREX also supports interface testing by recording and comparing database, message‑queue, and Redis interactions, ensuring no dirty data is produced during replay.

In production, AREX has significantly reduced manual test‑data creation and automation effort, especially for large‑scale data‑driven scenarios and frequent releases. Teams report higher test coverage, fewer missed defects, and faster self‑testing for refactoring projects.

Future work focuses on reducing false‑positive diffs through automated noise filtering, building a knowledge base of comparison rules, and tighter integration of code‑change tracing with test results to enable precise, observable debugging.