Moonbox: An Open‑Source Flow Recording and Replay Platform for Automated Testing

Author: vivo Internet Server Team - Liu Yanjiang

Moonbox is an automated testing platform based on traffic recording and replay, which replaces script writing for automated regression by recording and replaying, improving test efficiency and coverage. Because its solution is highly generic, we open‑source it to help users who need it.

1. What is Moonbox?

Moonbox (Moonlight Box) is a traffic recording and replay product under the JVM‑Sandbox ecosystem. Traffic recording and replay means the server automatically registers an agent probe, intercepts calls, and records all external call dependencies (such as databases, distributed caches, external service responses) to form a recorded traffic stream. Its core value is to transform recorded traffic data into reusable, executable automated test cases that can be replayed in a test environment to compare interface return values and external call parameters. Moonbox provides many common plugins to record and replay typical intermediate calls, as well as reliable, high‑performance data storage and computation capabilities.

2. What are Moonbox’s advantages?

Moonbox is a low‑threshold, high‑performance, easy‑to‑use automated testing tool for testing and development engineers. It has been running in vivo for over two years and has many practical, user‑friendly features. Its advantages include:

2.1 Comprehensive visual operation (partial features)

(1) Traffic management capabilities based on task and interface dimensions.

(2) Detailed traffic display (request, response, sub‑calls).

(3) Replay data management based on task and interface dimensions, with various statistical and query capabilities.

(4) Easy‑to‑analyze replay comparison results and difference display.

2.2 Rich plugin support

Moonbox supports a large number of component recording and replay capabilities, meeting the vast majority of user needs.

2.3 Multiple deployment methods

Docker: Simple and reliable, allowing you to discard complex environment configuration and installation, and quickly experience the project locally.

Conventional method: More complex and step‑heavy, requiring creation of ES and MySQL databases, initialization of tables, configuration updates, and installation of the front‑end node service.

In addition, Moonbox is a front‑back separation project; deploying it requires separate deployment of front‑end and back‑end, which facilitates production deployment.

2.4 Performance and safety reliability

The platform has undergone long‑term performance optimization and has been validated in multiple high‑concurrency systems within vivo. Security protections include limiting concurrent sampling per interface and restricting the number of parallel recording interfaces. The server uses Elasticsearch for traffic storage, effectively scaling data storage.

3. Implementation principle

3.1 Overall architecture

Moonbox consists of two parts: moonbox‑agent and moonbox‑server (architecture shown below).

moonbox‑agent

Uses Java‑attach technology (dynamic bytecode enhancement is implemented by JVM‑Sandbox) to dynamically attach to the target process, providing traffic recording and replay enhancements.

moonbox‑server

Agent side provides services such as configuration query, recording storage, traffic query, and replay result storage.

Configuration of recording/replay tasks, remote management of agent tasks, and a management UI (front‑back separation).

3.2 Traffic recording & replay

Traffic recording

The core logic distributes the agent to the machines specified by the user (local or remote), attaches the agent to the correct process of the target application, and intercepts key call points via JVM‑Sandbox’s BEFORE, RETURN, and THROW events to capture input and output parameters. The overall flow is shown below.

Traffic replay

The core logic distributes the agent to the machines specified by the user, attaches it to the correct process, and after the agent starts, it continuously pulls traffic from the server and replays it on the corresponding interfaces. The overall flow is shown below.

Heartbeat management

After the agent starts, it launches a separate thread that periodically sends HTTP heartbeat requests to the server.

3.3 Agent startup process

A script attaches the sandbox agent to the target Java process, starts a Jetty service, loads the Moonbox module, pulls Moonbox configuration from the server, and loads the recording and replay plugins.

4. Relationship with jvm‑sandbox‑repeater

Moonbox is redeveloped based on jvm‑sandbox‑repeater. Many modules have been rewritten, providing richer functionality and easier online deployment. The differences are illustrated below.

5. Why open source?

Traffic recording and replay technology is complex and challenging. Existing open‑source products often lack ease of use. By open‑sourcing Moonbox, we aim to help developers quickly build their own automated tools, allow customization, and give back to the community. Community feedback also helps us improve the tool and ensure its long‑term development.

6. Roadmap

Moonbox has released version V1.0.0 as open source, completing key functionalities. Future work includes performance and experience optimization, and incorporating community‑driven feature requests. Planned iterations for 2023 are shown below.

7. Final remarks

The open‑source release of Moonbox is just a small step. We will continue to contribute versions and features to the community. If you are interested, feel free to try the open‑source product, report issues, or submit pull requests on GitHub.

GitHub project address: https://github.com/vivo/MoonBox