How MaFengWo Scales AB Testing with OpenResty and Multi‑Layer Caching

What is ABTest

ABTest divides users into groups to run parallel product versions online, using real‑user feedback to decide which version performs better; the control group is the original version and the experiment iterates with small traffic before full rollout.

OpenResty Multi‑Layer Splitting Model

Instead of per‑service API calls that double traffic, the system uses OpenResty at the upstream layer to intercept requests and distribute traffic via HTTP or gRPC, avoiding second‑level traffic and keeping business code non‑intrusive.

Overall Process

When a request arrives, the AB gateway extracts device ID and URI, matches them against configured strategies, and returns the experiment and version either in an abtest header (for web) or a cookie (for micro‑services).

Stability Guarantee: MurmurHash

The traffic‑splitting algorithm uses MurmurHash on device ID, strategy ID, and layer ID; the hash is fast, distributes similar strings uniformly, and supports orthogonal and mutually exclusive experiments.

Traffic‑Layer Experiment Splitting

Within a traffic layer, hash factors (device ID, layer ID) are hashed with MurmurHash2, modulo 100 plus 1, yielding a number 1‑100 that determines which single experiment a user hits.

Experiment‑Version Splitting

Version allocation uses the same hash factors (device ID, strategy ID, layer ID) with identical matching rules to distribute traffic among version buckets.

Multi‑Level Cache Strategy

The strategy pool is cached in three layers: (1) an LRU cache bound to each Nginx worker, (2) a lua_shared_dict shared across workers, and (3) Redis as the persistent store. A lua‑resty‑lock ensures only one request fetches from Redis when both caches miss, protecting Redis from overload.

Key Features

Throughput : handles 5% of site traffic.

Low Latency : average response time under 2 ms.

Cross‑Platform : supports App, H5, WeChat Mini‑Program, PC.

Disaster Recovery : automatic downgrade on Redis failure and manual one‑click shutdown for high load.

Performance Results

Load tests with JMeter (100 concurrent users, 300 s) show response times mostly under 1 ms after warm‑up and stable TPS with only minor degradation due to hashing.

Conclusion

AB testing is most effective with sufficient traffic (≥ 1 000 UV per version). MaFengWo’s platform demonstrates that non‑intrusive traffic interception, layered caching, and hash‑based splitting can deliver high performance and flexibility for large‑scale experiments, while future work will focus on monitoring, user‑profile customization, statistical power analysis, and AARRR‑based impact evaluation.