Demystifying Clusters, Load Balancing, and Caching in Modern Backend Architecture

In the era of high concurrency, massive data, distributed systems, NoSQL, and cloud computing, many have heard of clusters and load balancing but rarely understand them deeply. This article provides a concise introduction to these concepts.

1: Evolution of Project Architecture

ORM and MVC: Early architectures ran on a single server, sufficient for small traffic. As business grew, MVC split the application into presentation, business, and data layers, simplifying development.

RPC Architecture: When a single server can no longer handle load, RPC distributes services across multiple machines, communicating via remote calls.

Service Provider: runs on a server, offering service interfaces and implementations.

Service Registry: runs on a server, publishing services, managing them, and providing them to consumers.

Service Consumer: runs on a client, invoking remote services through proxies.

Common Java RPC frameworks include Dubbo, Spring Cloud, and Thrift.

SOA Architecture: As services increase, RPC communication becomes complex. SOA introduces a service governance center to manage and register services, reducing chaos.

Micro‑services further decompose business logic into fine‑grained, independent services.

2: Terminology Explanation

The following terms often sound impressive to outsiders but are essential to understand.

1: Cluster

A cluster is a loosely coupled set of independent computers that communicate over a network to perform distributed computing, effectively working together as a single service.

Typical characteristics of large‑scale clusters include:

High Availability (HA): Backup servers automatically take over when the primary fails, ensuring uninterrupted service.

High Performance Computing (HP): Parallel processing of complex calculations, used in scientific fields such as genomics and chemistry.

Load Balancing (LB): Distributing workload across nodes to reduce pressure on any single server.

Common cluster types:

Load‑balance cluster – distributes tasks based on current workload.

High‑availability cluster – provides failover capability (active/standby, active/passive, active/active).

High‑computing cluster – multiple nodes collaborate to complete large tasks quickly.

2: Load Balancing

HTTP Redirect Load Balancing: The web server returns a 302 redirect to a different URL, directing the client to a nearer server. Simple but adds an extra request and can affect SEO.

DNS Load Balancing: A domain name resolves to multiple IP addresses, allowing DNS to distribute traffic. It can incorporate geographic routing but suffers from propagation delays and limited rule flexibility.

Reverse Proxy Load Balancing: A reverse‑proxy sits before web servers, caching resources and forwarding requests based on load‑balancing algorithms. It simplifies deployment but can become a performance bottleneck.

Load‑Balancing Strategies:

Round Robin

Weighted Round Robin

Least Connections

Fastest Response

Hash‑based

3: Caching

Caching stores data closer to the processor to accelerate access, a fundamental performance optimization.

CDN Cache: Content Delivery Networks cache static resources near end users, reducing latency for high‑traffic sites.

Reverse‑Proxy Cache: Positioned at the front of a website, it caches static assets before they reach application servers.

Local Cache: Application servers keep hot data in memory, avoiding database calls.

Distributed Cache: Large‑scale systems use a dedicated cache cluster accessed over the network to store data that exceeds a single machine’s memory.

4: Flow Control

Traffic Dropping: Simple queues may discard excess requests, which can be harsh for I/O‑intensive workloads.

More sophisticated approaches use distributed message queues to asynchronously process requests, improving resource utilization and user experience.

- End -