Handling High‑Concurrency Scenarios: Architecture, Caching, Rate Limiting, and Isolation Strategies

Background

In today’s rapidly evolving Internet environment, many enterprises face high‑concurrency challenges. As user numbers grow, the demand for handling massive simultaneous requests, system performance, stability, and fault tolerance also increases, putting pressure on architecture, database design, caching strategies, automated operations, security, cost, and efficiency.

Marketing Rights Platform (Spring Festival Gala 2024)

The platform handled lottery‑type coupon distribution and interactive wallet balance allocation, providing technology‑coupon verification, claim, and query capabilities for the event.

1. Rights Verification Process

The verification function determines whether a user can claim a specific right. When a user clicks “Start Lottery,” the retail UMC calls the verification API to batch‑retrieve eligible rights, performing checks such as basic info, inventory, claim rules, eligibility, and risk control before returning the list of claimable rights.

2. Rights Issuance Process

After verification, UMC obtains all eligible rights (coupons, red packets, vouchers, physical items) and, based on strategy, forwards the specific rights to SOA. SOA then invokes the issuance API, which performs basic, rule, and risk checks, deducts inventory and user qualification, writes the rights to the database, and returns the final claim result.

Challenge

Understanding that MCube first checks template cache status to decide whether to fetch the latest template from the network; after loading, the product is transformed into a view‑tree, expressions are evaluated, custom events are parsed and bound, and finally the view is rendered.

1. Cost and Service

Balancing cost, service quality, and business innovation is the biggest architectural challenge.

2. Three‑High Guarantees

Meeting high performance, high concurrency, and high availability simultaneously, while middleware and data services become bottlenecks that cannot be solved merely by adding machines.

Implementation

Similar to the challenge description, MCube processes templates, parses expressions, binds events, and renders the view.

1. Rate Limiting

Proper rate limiting is crucial for system stability and resource utilization; the evaluation priority is user experience > business goals > system pressure.

1) Business‑Driven Analysis

Clarify business scenarios and user‑experience impact when throttling occurs.

Define business goals, e.g., total coupons to be issued during the gala.

Identify traffic characteristics, such as peak‑time patterns, to assess impact of shedding load.

2) Historical Data Analysis

Use data from daily marketing, promotions, and events to drive the selection of an appropriate throttling value.

2. Caching (Heavy Cache)

Using cache significantly improves performance, reduces data‑source pressure, and enhances user experience. The marketing system employs both local and Redis caches.

Local cache can be built in two ways:

Level‑1 cache: local + MySQL + push‑mode updates.

Level‑2 cache: local + r2m + MySQL + push‑mode updates.

Choosing between them depends on cluster size, data volume, database performance, and trade‑offs between space and time.

3. Merging

Designing business flows to allow parallel processing via multithreading and batch requests reduces network overhead and I/O, protecting downstream systems while maintaining performance.

4. Splitting

Splitting is a common solution that trades space for time. Inventory hotspots during the gala are addressed by splitting strategies.

1) Traditional Method (Daily Coupon Path)

Read path uses GET, write path uses Lua for inventory control, supporting ~40k QPS (read+write). Suitable for low‑traffic, high‑consistency scenarios.

2) Local Inventory Method (Cashier Path)

Read path pre‑fetches inventory locally for deduction; write path uses Lua, supporting up to 20k writes with unlimited reads. Strong consistency, large inventory, read‑heavy scenarios.

3) Sharded Inventory (Gala Version)

Read path checks local inventory flags; write path shards inventory by pin using hashtag access, offering weak consistency but high read/write concurrency for large‑scale scenarios.

5. Isolation

Isolation improves security, fault tolerance, performance protection, and manageability.

Same‑datacenter calls: jsf, jmq, r2m aim for same‑datacenter communication; during the gala, jmq spans retail and finance datacenters with services deployed in finance to reduce latency.

Data isolation: user qualification, rights records, and claim logs are stored separately, with gala‑specific coupon status isolated.

Read‑write separation: core r2m and cds are separated to reduce primary node pressure.

Business isolation: asynchronous persistence messages are isolated to prevent backlog from delaying other activities.

Conclusion

Solutions for high‑concurrency scenarios must consider architecture, technology selection, performance optimization, and capacity planning, continuously adapting to evolving business needs while emphasizing fault tolerance and security to ensure system stability and reliability.

Recommended Reading

Inventory Core Capability – Pre‑Reservation Practice

JD Advertising R&D – Efficiency First: Unified Search Platform

JD Mid‑Platform Framework Technical Analysis

AIGC Visual Applications in Insurance