Mastering Distributed System Design: Patterns, Performance, and Fault Tolerance

Design Patterns

Gateway (API Gateway)

Routes client requests to registered backend services.

Maintains a service registry so each service can publish its API endpoints.

Provides load‑balancing strategies such as round‑robin, random, weighted, session‑sticky and custom algorithms.

Enforces security (HTTPS, authentication, DDoS protection).

Supports canary releases, API aggregation and orchestration.

Sidecar

Runs alongside a service to offload cross‑cutting concerns (routing, flow control, circuit breaking, idempotency, service discovery, authentication).

Typical for legacy‑system refactoring, multi‑language micro‑service extensions, or composite applications.

Design guidelines: use language‑agnostic protocols (HTTP/TCP), avoid intrusive IPC (e.g., shared memory), keep sidecar‑to‑service contracts stable.

Service Mesh

A lightweight network‑proxy layer that abstracts inter‑service communication, making services unaware of transport details.

Key implementations: Istio , Linkerd .

Performance Techniques

Caching Strategies

Cache‑Aside : Application explicitly reads/writes the cache and handles invalidation.

Read/Write‑Through : Cache sits in front of the data store; all reads/writes go through the cache, presenting a single storage view.

Write‑Behind : Writes are buffered in the cache and persisted to the database asynchronously, reducing write latency.

Asynchronous Communication

Push model : A central scheduler pushes tasks to workers (higher coordination complexity).

Pull model : Workers poll a task queue (simpler implementation).

Hybrid Push‑Pull : Combines both for flexibility.

Event‑Driven Architecture (EDA) : Producers publish events to a broker (e.g., RocketMQ); consumers subscribe and process events, achieving loose coupling and high isolation.

Database Sharding

Vertical sharding : Split tables by columns with distinct access patterns.

Horizontal sharding : Distribute rows using hash‑based or time‑range partitioning.

Design tips: reserve identifier space for future splits, parallelize aggregation queries, avoid cross‑shard transactions.

Fault Tolerance

Rate Limiting

Purpose: enforce SLA, protect against traffic spikes, provide tenant isolation.

Algorithms: token bucket, leaky bucket, fixed‑window counters, weighted queues.

Operational considerations: manual toggle, alerting, user‑visible error codes, RPC‑level tags.

Circuit Breaker

States: Closed (normal), Open (reject all), Half‑Open (test limited traffic).

Key practices: define error types that trigger the breaker, log every tripped error, enable automatic recovery, optionally provide a manual switch, isolate failures per business domain.

Compensation Transactions

Contrast CAP (Consistency, Availability, Partition tolerance) with BASE (Basic Availability, Soft state, Eventual consistency).

Failure‑oriented design: use exponential back‑off, make operations idempotent.

Additional Distributed System Primitives

Distributed Lock

Implementation using Redis SETNX key value PX expireTime where value is a globally unique token (e.g., UUID or TraceID).

Expiration is in milliseconds; the lock auto‑releases if the holder does not complete within the TTL.

Variants: pessimistic lock (acquire before operation, low throughput) and optimistic lock (version number, high read concurrency).

Design requirements: exclusivity, automatic release, high availability & persistence, non‑blocking & re‑entrant, dead‑lock avoidance, cluster fault tolerance.

Idempotency

Guarantees that repeated execution of an operation yields the same result.

Common unique identifiers: auto‑increment DB IDs, locally generated UUIDs, Redis‑generated IDs, Snowflake algorithm.

HTTP methods except POST (GET, HEAD, OPTIONS, DELETE, PUT) are inherently idempotent.

Configuration Center

Static configuration : values set at application start (environment variables, config files).

Dynamic configuration : runtime‑adjustable switches (feature flags, flow‑control toggles).

DevOps Practices

Deployment Strategies

Stop‑the‑world (full downtime), rolling update, blue‑green, canary release, A/B testing.

Configuration Management

Tools: Ansible , Puppet , Shippable for declarative infrastructure and feature‑flag distribution.

Monitoring

Observability stacks: Nagios , Dynatrace for health checks, latency, error rates.

CI/CD

Continuous Integration: Jenkins , CodeShip for automated builds and tests.

Continuous Delivery: pipelines that automatically promote artifacts through staging to production after passing quality gates.

Key Metrics for Availability

MTTF (Mean Time To Failure): average uptime before a failure; higher is better.

MTTR (Mean Time To Recover): average time to restore service; lower is better.

Availability formula: Availability = MTTF / (MTTF + MTTR).