Designing High-Concurrency Systems: Key Strategies and Best Practices

1. Page Static‑Generation

Dynamic rendering under massive traffic can overload servers; converting pages to static HTML using template engines such as Freemarker or Velocity reduces load. For a storefront homepage, a scheduled Job gathers data, renders it to an HTML file, and a shell script syncs the file to web servers.

2. CDN Acceleration

Static pages still suffer from geographic latency. A Content Delivery Network (CDN) copies static assets (images, CSS, JS) to edge nodes worldwide, serving users from the nearest node, thus lowering network congestion and improving response time.

3. Caching

Caching is indispensable. Two main types are:

Application‑server memory cache (second‑level cache).

Distributed cache middleware such as Redis or Memcached.

Second‑level caches are faster but can cause data inconsistency across multiple nodes; distributed caches avoid this at the cost of slightly lower performance. Examples show how caching reduces database pressure and improves throughput, and how combining both caches can serve scenarios like product‑category retrieval.

4. Asynchronous Processing

Non‑core operations (e.g., sending internal notifications, writing logs) should be asynchronous. Two approaches:

4.1 Thread Pool

Core business logic runs synchronously; auxiliary tasks are submitted to dedicated thread pools, instantly boosting interface performance. However, server restarts or task failures can lead to data loss.

4.2 Message Queue (MQ)

Interface sends MQ messages; consumers execute the auxiliary logic. This also improves latency while providing better reliability than raw thread pools.

5. Multi‑Threaded Consumption

When MQ producers generate a flood of messages faster than consumers can process, message backlog occurs. Switching consumers to a configurable thread pool (tuning core size, max size, queue length, and keep‑alive) alleviates the bottleneck.

6. Database Sharding & Partitioning

High traffic can exhaust DB connections and I/O. Splitting data across multiple databases (vertical) and tables (horizontal) solves resource limits. Routing algorithms include modulo by ID, range partitioning, and consistent hashing. Vertical sharding isolates business domains; horizontal sharding distributes rows to balance load.

7. Connection Pooling

Creating and destroying DB connections per request is costly. Pooling (e.g., Druid, C3P0, Hikari, DBCP) reuses connections, reducing latency and CPU usage.

8. Read/Write Splitting

Since reads dominate writes (≈80/20), separating read traffic to slave replicas prevents read‑write contention. A master‑slave or master‑multiple‑slaves topology distributes load and provides failover paths.

9. Index Optimization

Indexes accelerate queries on large tables but add overhead to inserts. Best practices: use composite indexes where appropriate, drop unused indexes, analyze execution plans with EXPLAIN, avoid index‑invalidating patterns, and force specific indexes when needed.

10. Batch Processing

Fetching many rows individually incurs many remote calls. Refactor code to batch‑query IDs in a single SQL statement. Example:

public List<User> queryUser(List<User> searchList) {
    if (CollectionUtils.isEmpty(searchList)) {
        return Collections.emptyList();
    }
    List<Long> ids = searchList.stream().map(User::getId).collect(Collectors.toList());
    return userMapper.getUserByIds(ids);
}

Limit batch size (e.g., ≤500) based on actual workload.

11. Clustering

Deploy multiple server nodes (application, DB, middleware, file servers). For Redis with 40 GB data and 16 GB per node, use three master nodes with sharding and a slave for each to provide redundancy.

12. Load Balancing

Distribute requests across nodes using software (Nginx, LVS, HAProxy, Ribbon, OpenFeign, Spring Cloud LoadBalancer) or hardware (F5). Strategies include round‑robin, weighted, IP‑hash, least‑connections, and shortest‑response‑time.

13. Rate Limiting

Prevent abuse and protect stability with:

User‑level limits (e.g., 5 requests/min).

IP‑level limits.

Interface‑level limits.

CAPTCHA (including sliding‑block CAPTCHA) for precise control.

14. Service Degradation

Expose switches (e.g., via Apollo) to disable non‑essential features (like product comments) during overload. Use circuit‑breaker libraries such as Hystrix or Sentinel to fallback when downstream services fail.

15. Failover

Detect unhealthy nodes (high CPU, timeouts) and automatically route traffic to healthy instances. Configure Ribbon health checks and Hystrix thresholds to trigger failover.

16. Multi‑Active Across Data Centers

Deploy the system in multiple regions (e.g., Shenzhen, Tianjin, Chengdu) with traffic split (40/30/30). DNS routing directs users to the nearest data center; data synchronization ensures consistency.

17. Stress Testing

Estimate peak QPS, then run load tests (JMeter, LoaderRunner, Locust, Alibaba PTS) to determine required node count (e.g., 10 000 QPS → 10 nodes at 1 000 QPS each, then over‑provision by 3× for safety).

18. Monitoring & Alerting

Use Prometheus to collect metrics such as response latency, third‑party call time, slow‑query duration, CPU/memory/disk usage, and DB statistics. Visual dashboards help spot resource saturation, connection leaks, or cache‑related issues.

19. Security & Resilience Considerations

Address IP‑fluctuating attacks, cache‑snowball effects, DDoS mitigation, and dynamic scaling to handle traffic spikes.