High Concurrency and High Availability Design Principles

1. High Concurrency Principles

1.1 Stateless

If an application is designed to be stateless, it can be easily scaled horizontally; in practice, the application itself is stateless while configuration files may retain state.

1.2 Decomposition

When traffic is large and resources are sufficient, consider decomposing the system. Main decomposition dimensions include:

System level: split by functional or business modules.

Feature level: split a system by individual features.

Read/Write level: use caching for heavy reads, sharding or partitioning for heavy writes; aggregate read scenarios may use heterogeneous data stores to improve performance and reliability.

AOP level: split based on access characteristics using AOP.

Module level: split according to code maintenance or foundational concerns.

1.3 Serviceization

In‑process service → remote single‑machine service → manually registered cluster service → automatic registration and discovery → service grouping/isolation/routing → service governance (rate limiting, black‑/white‑list).

1.4 Message Queues

Message queues decouple services that do not require synchronous calls, enable one‑to‑many consumption, asynchronous processing, and traffic smoothing/buffering.

1.5 Data Heterogeneity

1.5.1 Data Heterogeneity

Order tables are often sharded by order ID; querying a user's orders requires aggregating multiple tables, leading to low read performance. Heterogeneous data storage can create a user‑order table sharded by user ID, improving performance.

Archiving order data also helps improve service performance and stability.

1.5.2 Data Closed‑Loop

For pages like product details that gather data from many sources, the best approach is to store heterogeneous data and form a closed‑loop. Steps:

Data heterogeneity: receive changes via MQ and atomically store them in appropriate stores such as Redis or persistent KV.

Data aggregation: aggregate data from multiple sources, typically stored in a KV store for front‑end consumption.

Front‑end presentation: the front‑end fetches required data in one or few calls.

This method ensures that even if a dependent system fails, the front‑end can still display data, though updates may be delayed.

When multiple data items are needed simultaneously, a HashTag mechanism can co‑locate related data in the same instance (e.g., using productId as part of the key).

1.6 Cache "Silver Bullet"

Browser cache

App client cache

CDN cache

Edge layer cache

Application layer cache

Distributed cache

For fallback or abnormal data, caching should be avoided to prevent prolonged exposure of incorrect information to users.

1.7 Concurrency

Parallelize serial behaviors.

2. High Availability Principles

2.1 Degradation

Design a degradation switch for high‑availability services. Key ideas:

Centralized switch management: push switches to all applications.

Multi‑level degradable read services: fallback to local cache, distributed cache, or default data (e.g., assume inventory is in stock).

Switch pre‑positioning: place switches at the Nginx layer before the application.

Business degradation: during traffic spikes, prioritize order placement and payment, possibly converting synchronous calls to asynchronous ones and handling high‑priority data first.

2.2 Rate Limiting

Limit traffic to prevent malicious requests, attacks, or exceeding system peaks. Approaches include:

Direct malicious traffic to cache only.

Use Nginx limit module for traffic reaching back‑end services.

Block malicious IPs with Nginx deny rules.

The principle is to restrict traffic from reaching vulnerable application layers.

2.3 Traffic Switching

For large applications, traffic switching is crucial when a data center, rack, or server fails. Methods include DNS, HttpDNS, LVS/HaProxy, and Nginx.

2.4 Rollback

Versioning enables auditability, traceability, and rollback. When code or data errors occur, rollback mechanisms (transaction, code repository, deployment, data, static resources) help restore a correct state, supporting high availability.

3. Business Design Principles

3.1 Idempotent Design

Prevent duplicate payments, duplicate deductions, etc.

3.2 Idempotent Operations

An idempotent operation yields the same effect regardless of how many times it is executed with the same parameters.

3.3 Configurable Processes

Reuse a workflow system to provide personalized process services.

3.4 State and State Machines

Order systems have forward states (awaiting payment, shipping, completed) and reverse states (cancellation, refund). State design should include state trajectories for tracking and logging, enabling issue traceability.

3.5 Backend Operations Feedback

Design backend systems with preview and feedback capabilities.

3.6 Backend Approval Workflow

Critical backend functions (e.g., price adjustments) should have approval flows and audit logs.

3.7 Documentation and Comments

Early-stage systems need documentation libraries (architecture, design ideas, data dictionary, business processes, known issues) and code comments for special requirements.

3.8 Backup

Backup code via version‑controlled repositories and ensure at least two developers understand each system.

4. Summary

System design must not only fulfill business functions but also ensure high concurrency, high availability, and reliability. It should consider capacity planning, SLA definitions, monitoring and alerts, and emergency plans such as disaster recovery, degradation, rate limiting, isolation, traffic switching, and rollback.

4.1 High Concurrency Principles

Cache

Asynchronous concurrency

Connection pools

Thread pools

Scaling

Message queues

Distributed tasks

4.2 High Availability Principles

Load balancing and reverse proxy for traffic splitting.

Rate limiting to protect services from avalanche failures.

Degradation to provide partial, lossy service.

Isolation for fault containment.

Proper timeout and retry settings to avoid request buildup.

Rollback mechanisms for quick error correction.