Load Balancing, Reverse Proxy, and Isolation Techniques

1. Load Balancing and Reverse Proxy

When a single application instance cannot handle user traffic, scaling out to multiple servers is required; users still access the service via a single domain name, which resolves to an IP address through DNS.

Key aspects of load balancing include upstream server configuration, load‑balancing algorithms, failure‑retry mechanisms, and server health‑check heartbeats.

Nginx provides load‑balancing, failover, retry, fault‑tolerance, and health‑check capabilities, and can act as a reverse proxy that forwards client requests to internal upstream servers while optionally caching or compressing responses for performance.

Load‑Balancing Algorithms

round-robin : default algorithm that distributes requests sequentially; can use weights for weighted round‑robin.

ip‑hash : routes requests from the same client IP to the same upstream server.

hash key (consistent) : hashes a specific key or uses consistent hashing to minimise key redistribution when servers are added or removed.

custom hash : uses request URI or Nginx variables for more complex hashing strategies.

Failure retry is controlled by max_fails and fail_timeout ; a server exceeding the failure threshold is temporarily removed from the pool and reinstated after the timeout.

Health checks are lazy by default; the commercial Nginx version offers active health checks, and the nginx_upstream_check_module can perform active TCP/HTTP heartbeats.

HTTP Dynamic Load Balancing

Consul, an open‑source service‑registry and discovery system, provides HTTP APIs for service registration, discovery, health checking, key‑value storage, multi‑data‑center support, and uses the Raft algorithm for consistency.

2. Isolation Techniques

Isolation separates systems or resources to limit fault propagation and resource contention, ensuring that failures affect only a bounded scope.

Thread Isolation

Thread‑pool isolation assigns different request categories to separate thread pools, preventing a failure in one business flow from affecting others.

Process Isolation

Splitting a monolithic system into multiple sub‑systems (processes) ensures that a crash or OOM in one does not bring down the entire application.

Cluster Isolation

Deploying services as independent clusters (e.g., a dedicated cluster for flash‑sale traffic) isolates hot‑spot workloads and prevents them from impacting other services.

Data‑Center Isolation

Multi‑data‑center deployments keep services within their own zone; failures in one zone are mitigated by DNS or load‑balancer routing to another zone.

Read/Write Isolation

Using master‑slave replication separates read traffic from write traffic; if the master fails, read replicas remain available.

Static/Dynamic Isolation

Static assets (JS, CSS, images) are served from CDNs, separating them from dynamic application logic to avoid bandwidth contention.

Crawler Isolation

Crawler traffic is routed to a dedicated cluster via load balancing, preserving capacity for normal user traffic.

Hotspot Isolation

High‑traffic features such as flash sales are implemented as independent services to contain failures.

Resource Isolation

CPU, memory, disk, and network resources are partitioned to prevent one service from exhausting shared resources.

Hystrix for Isolation

Hystrix, a Netflix open‑source library, provides thread and semaphore isolation, circuit‑breaker, and graceful degradation to prevent cascading failures and improve system resilience.

Reference: Zhang Kaitao, *Core Technologies of Billion‑Scale Web Architecture*.