Beyond Cache+Hash: Real Strategies for Building High‑Concurrency Systems

Cache+Hash Myth

Many high‑concurrency systems are oversimplified as merely "cache + hash", a notion that originated from e‑commerce flash‑sale traffic and has become a buzzword across the internet.

Typical High‑Concurrency Techniques

Static Resource Serving – Use Nginx or CDN to serve static pages; if CDN is unavailable, rely on LVS/F5 load‑balancing to distribute traffic.

Read‑Write Separation & Sharding – Primary database handles writes while replicas serve reads; further split data across multiple databases using hash‑based sharding to reduce per‑node load.

Universal Caching – Deploy mature KV stores such as memcache or redis in cluster mode; they implement the caching layer but are not the whole solution.

Hashing – Hash tables provide O(1) insert and lookup, making them a staple in high‑throughput designs.

Real‑World Examples

Flash‑sale page : Simple logic, massive QPS; success depends on bandwidth, Nginx static serving, and aggressive cache‑hash usage.

Ticketing system (e.g., 12306) : Extremely high traffic but limited resources (e.g., 1000 tickets); requires rate‑limiting, queuing, and asynchronous processing beyond pure cache‑hash.

Designing High‑Concurrency Systems

The key is not to copy patterns blindly but to choose appropriate data structures.

1. Reasonable Data Structures

Search‑suggestion services illustrate this point. Instead of storing every suggestion in Redis, a Trie (prefix tree) offers O(k) lookup where k is the query length, eliminating network overhead.

2. Continuous Code‑Level Optimization

After selecting a data structure, profile the service, identify hotspots, and iteratively refactor code. Tools like go tool pprof (for Go) or Java profilers help pinpoint bottlenecks.

3. External Generic Solutions

Only after exhausting internal optimizations should you consider adding generic caches (e.g., LRU) or other third‑party components.

4. Operations & Deployment

Scale out with load balancers, clusters, and proper CI/CD pipelines to handle further traffic growth.

Time‑Space Trade‑offs in Trie Implementations

Basic Trie – Simple 26‑ary array per node (for English); memory intensive for Chinese characters.

Optimized Variable‑Length Trie – Replace fixed arrays with dynamic lists or binary search, reducing memory at the cost of O(log n) lookup per node.

Double‑Array Trie – Uses base and check arrays to represent a finite‑state machine; offers compact storage and fast lookup but complicates traversal for suffix suggestions.

Choosing among them depends on dataset size, query latency requirements, and whether suffix traversal (e.g., retrieving related terms) is needed.

Offline vs. Online Processing

Search‑suggestion pipelines typically perform log cleaning, hot‑term filtering, and Trie construction offline, updating the service daily or weekly. Offline sorting of suggestions yields faster response but consumes more storage; online scoring reduces space at the expense of a tiny runtime sort (usually <10 items).

Overall Guidance

If the query set is stable and performance‑critical, use a double‑array Trie with offline construction and full sorting.

If the dataset grows rapidly or memory is constrained, adopt an optimized variable‑length Trie with offline scoring and runtime top‑k sorting.

For modest traffic, a simple Redis key‑value cache may suffice as a quick prototype.

Ultimately, architecture decisions should balance time (CPU, latency) and space (memory, storage) based on concrete workload characteristics rather than following hype.