Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience

Microservice Decomposition

Microservices form the core of large‑scale architectures; the primary goal is to split a complex system into vertically isolated, independently deployable services based on business functions.

Each service should be developed, deployed, and scaled independently, have a single responsibility with clear interfaces, and communicate via lightweight protocols such as HTTP, gRPC, or message queues. This eliminates monolithic performance bottlenecks and enables each service to scale to meet high‑concurrency demands.

Business and Data Boundaries

Every service must own a well‑defined business and data boundary, preferably maintaining its own database or storage to avoid cross‑service write conflicts and tight coupling.

High‑frequency paths—such as order placement, payment processing, and inventory updates—should be prioritized for splitting so they can be scaled and optimized independently.

Data Sharding Strategies

When a single database or table reaches tens of millions or billions of rows, query performance degrades, lock contention rises, and disk I/O becomes a bottleneck. At this point, data‑layer sharding is required.

Vertical Sharding (Database Splitting) – Divide a large database into multiple databases per business module, e.g., user_db, order_db, product_db, each deployed independently to distribute I/O load.

Horizontal Sharding (Table Splitting) – Split large tables into multiple tables based on rules:

By user ID modulo: order_0, order_1, order_2, …

By time period: order_2025_10, order_2025_11, …

Service Rate Limiting

Rate limiting protects the system under heavy traffic, preventing avalanche failures, thread‑pool exhaustion, database crashes, and full‑chain outages. It is a key component of the “high‑availability four‑pillars” together with circuit breaking, degradation, and isolation.

Service Circuit Breaking

When a downstream service becomes slow or fails, callers must stop waiting indefinitely to avoid cascading failures. Circuit breakers trigger when failure or timeout rates exceed configured thresholds (e.g., 50% failure rate with a minimum of 20 requests per window).

Typical implementations include:

Hystrix – classic Netflix solution.

Resilience4j – lightweight modern library.

Sentinel – Alibaba’s open‑source distributed flow control and circuit‑breaker system.

Service Degradation (Fallback)

Even when some services are unavailable, the system should continue operating in a “degraded but usable” mode. Degradation strategies include returning default data, hiding non‑core features, or queuing requests for later processing.

Examples:

Product detail fallback – if the review service times out, show only product information.

Payment fallback – on payment gateway failure, persist the order for later settlement.

Recommendation fallback – when the recommendation engine is down, return popular items.