Scalable Architecture for Community & E‑Commerce: Sharding, Caching & Distributed Transactions

1. Community System Architecture

Using DDD domain models, the monolithic system is split into multiple microservices based on Spring Cloud, aiming for minimal fan‑out and maximal fan‑in, client‑side load balancing, and upstream rate limiting and degradation. Each service should stay around 10k lines of code to maintain simplicity.

Typical multi‑stage decomposition separates modules into distinct subsystems such as Order, Product, Procurement, Warehouse, and User services, which may further split (e.g., Procurement → Supplier Management, Purchase Order Management; Order → Cart, Pricing, Order Management).

2. E‑Commerce System – Billion‑Level Product Storage

Data is partitioned using hash modulo and consistent hashing for both sharding databases and tables. High‑concurrency reads are handled by multi‑level caching (Redis cluster, local cache, rate limiting, random key suffixes). Writes use the same sharding strategy to evenly distribute load.

Hot‑key mitigation can be achieved by range‑based sharding to disperse hotspot sub‑keys.

3. Reconciliation System – Distributed Transaction Consistency

Avoid distributed transactions when possible; use local DB transactions for single‑process operations and message queues for cross‑process consistency.

Eventual consistency via reliable MQ delivery.

Maximum‑effort notification with retry‑and‑acknowledge.

Other theoretical solutions (2PC, 3PC, TCC, SAGA) are costly in practice and only suitable for strong‑consistency scenarios such as financial payments.

4. User System – Multi‑Threaded Data Migration

During a large‑scale user data migration, duplicate processing occurred due to concurrent threads sharing mutable lists. Replacing ArrayList with CopyOnWriteArrayList did not solve the issue; the root cause was clearing the shared list before child threads finished processing.

Solution: deep‑copy the list outside the thread, pass the copy to the child thread, and perform clear only after the child thread completes.

if (arrayBuffer.length == 99) {
    val asList = arrayBuffer.toList
    exec.execute(openIdInsertMethod(asList))
    arrayBuffer.clear
}

5. Statistics System – Massive Counting

For low‑to‑medium scale counting, a cache‑plus‑DB approach works, but high‑frequency counters (e.g., feed reads) quickly overwhelm databases. Large‑scale systems store counters entirely in Redis clusters using hash partitioning, master‑slave replication, and read‑write separation.

Memory efficiency is low (≈65 bytes per 4‑byte counter), so storing billions of counters requires massive memory; hybrid solutions keep hot keys in memory and cold keys on SSD with LRU eviction.

6. System Design – Microsoft Example

Key design steps include requirement gathering, top‑level design, defining core metrics (performance, latency, scalability, throughput, availability, consistency), and selecting appropriate storage: Redis for hot data, MongoDB for documents, Elasticsearch for search, HBase/BigTable for columnar big data, Neo4j for graph, FastDFS for media.

7. How to Design a Micro‑Blog

Core features: posting, timeline, news feed, follow/unfollow, registration/login. Capacity planning based on QPS (100 → single server, 1 K → robust single server with failover, 1 M → 1 000‑node cluster). Choose storage per service (SQL for relational data, Redis for high‑QPS counters, NoSQL for large‑scale reads). Use microservice decomposition, appropriate caching layers, and ensure idempotency and reliable MQ consumption.