Demystifying Clusters, Load Balancing & Caching in Modern Backend

In an era filled with buzzwords like high concurrency, massive data, distributed systems, NoSQL, and cloud computing, many have heard of clusters and load balancing but may not truly understand them.

Understanding these concepts starts with the evolution of project architecture.

1: Evolution of Project Architecture

ORM and MVC : Early architectures ran on a single server, sufficient for small traffic. Introducing MVC split the application into presentation, business, and data access layers, making development and maintenance easier.

RPC Architecture : As traffic grew, a single server became insufficient. RPC distributed architecture breaks services into separate components deployed on multiple servers, communicating via remote calls.

Service Provider: runs on a server, offers service interfaces and implementations.

Service Registry: runs on a server, publishes local services as remote, manages them, and provides them to consumers.

Service Consumer: runs on a client, invokes remote services through proxies.

Common Java RPC frameworks include Dubbo, Spring Cloud, and Thrift.

SOA Architecture : With further growth, RPC services proliferate, creating complex dependencies. SOA centralizes service management with a governance hub where services register and consumers discover them.

Microservices : The latest trend further decomposes business logic into fine‑grained, independently deployable services.

2: Terminology Explained

Below are concise explanations of terms that often sound impressive to outsiders.

1: Cluster

Cluster (Cluster) : A loosely coupled multiprocessor system composed of independent computers that communicate over a network, allowing distributed computation and shared memory messaging.

Large‑scale clusters typically provide:

(1) High Availability (HA) : Failover mechanisms ensure continuous service when the primary server fails.

(2) High‑Performance Computing (HP) : Parallel processing of complex tasks, common in scientific workloads.

(3) Load Balancing (LB) : Distributes workload across nodes to reduce pressure on any single server.

Common cluster types include:

Load‑balance cluster : Distributes tasks among multiple workers based on a policy, like a master delegating orders to several brothers.

High‑availability cluster : Provides standby or active/passive configurations to ensure uninterrupted service.

High‑performance computing cluster : Multiple nodes collaborate on a large, complex job, akin to many brothers jointly building a piece of furniture.

2: Load Balancing

HTTP Redirect Load Balancing : Uses HTTP 302 redirects to send clients to alternative URLs, simple but adds latency and can affect SEO.

DNS Resolution Load Balancing : Maps a domain name to multiple IP addresses, allowing DNS to distribute traffic, often with geo‑location routing.

Reverse Proxy Load Balancing : Places a reverse‑proxy server in front of web servers, forwarding requests based on algorithms and optionally caching content.

Load Balancing Strategies :

Round Robin

Weighted Round Robin

Least Connections

Fastest Response

Hashing

3: Caching

Caching stores data closer to the processor to accelerate access, a primary performance optimization technique.

CDN Caching : Distributes static resources to edge nodes near users.

Reverse Proxy Caching : Caches static assets at the front‑end proxy, reducing load on application servers.

Local Caching : Keeps hot data in the application server’s memory.

Distributed Caching : Uses a dedicated cache cluster to store large datasets beyond a single machine’s capacity.

4: Flow Control (Traffic Control)

Traffic Dropping : Simple approach that discards excess requests when queues are full, suitable for CPU‑bound workloads.

More sophisticated solutions involve asynchronous processing via distributed message queues.