High-Fidelity Load Testing: Practices and Technical Innovations from JD's ForceBot Platform

High-fidelity load testing is a practice that simulates or replays real traffic to evaluate production clusters, aiming to obtain results close to real‑world conditions by considering environment, traffic volume, and call chains.

1. Definition and criteria – High-fidelity testing reproduces production traffic through simulation or real‑traffic replay, requiring the test environment to match production, distinct traffic routing configurations, and accurate data handling such as traffic recording, business ratio modeling, and various testing modes (traffic replay, simulation, and back‑pressure testing).

2. Technical breakthroughs

2.1 Massive traffic generation platform – The ForceBot platform is a distributed performance testing system capable of constructing tens of millions of concurrent requests. It features a horizontally scalable architecture, containerized deployment, real‑time result computation, and has proven to support at least ten million requests during JD's 618/Double‑11 preparations.

2.2 Full‑domain traffic recording and replay

Recording real user traffic is essential for high-fidelity scenarios. For public‑network traffic, core switch ports are split: one path continues to the production service, the other feeds a recording service that captures, desensitizes, encrypts, and stores traffic files in a cloud file system (CFS) for later replay. For internal‑network traffic, agents are injected into application servers to capture traffic into MQ, cache, or cloud storage, supporting multiple protocols such as HTTP and RPC.

These recordings undergo desensitization and encryption to ensure data security before being replayed by the testing platform, which mounts the stored files directly onto pressure machines for efficient distribution.

3. Practice in 618/Double‑11 drills

The iterative upgrades of the testing technology have significantly improved testing efficiency and result reliability. The platform now supports simultaneous load testing across multiple IDC data centers without impacting production traffic, achieving a ten‑fold increase in traffic volume (from millions to tens of millions), expanding high‑fidelity coverage to 80% of critical systems, and being adopted by roughly 90% of JD’s development teams.