Why Distributed Systems Are Essential for Scaling Modern Web Services

Distributed Systems Increase Capacity: Basic Techniques

When an internet service becomes popular, a single server cannot handle millions of daily users, so developers must use multiple machines to provide the same application – the essence of a distributed system.

Key performance requirements are high throughput, high concurrency, low latency, and load balancing. High throughput means serving many users simultaneously, which requires multiple servers working together without bottlenecks. High concurrency extends throughput by ensuring each server works efficiently without unnecessary waiting. Low latency demands fast response even under heavy load, requiring careful request routing and minimizing the number of forwarding hops.

Because users are worldwide, they connect from different networks and time zones, so servers must be deployed in multiple locations and requests must be balanced across them.

Layered Model (Routing, Proxy)

The simplest approach is a polymorphic server pool where every server can handle any request. Early DNS round‑robin is an example: a domain name resolves to multiple IPs, and the client is sent to a random server.

Random routing fails for stateful services such as login. After a user logs in, subsequent requests must be directed to the same backend that holds the session. Therefore an additional layer examines cookies or credentials and forwards the request to the appropriate logical server.

Data that must be stored centrally (e.g., a database) is often isolated on dedicated servers. This leads to the classic three‑tier architecture: access layer, logic layer, and storage layer.

In interactive services like online games, the logic layer must coordinate user state across servers, often requiring a dedicated interaction server that records where each user is logged in and forwards messages accordingly.

Concurrency Model (Multithreading, Asynchronous)

Server programs must handle many simultaneous requests. A naïve single‑threaded design would waste time waiting for I/O (database, other services). Two main solutions exist:

Multithreading / multiprocess : easy to code but introduces race conditions, requiring locks that can cause deadlocks and increase overhead.

Asynchronous, non‑blocking I/O : uses callbacks and mechanisms like Linux epoll to handle many connections in a single thread, eliminating lock contention and context‑switch costs.

int epoll_create(int size); // create an epoll handle, size tells the kernel the number of fds to monitor
int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);
int epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout);

Buffering Technology

To reduce database load, systems cache query results in fast memory stores. Memcached is a classic example: web applications read from the cache first, falling back to the database only when necessary.

Memcached itself lacks built‑in clustering, so developers must manually distribute keys across multiple cache instances. More advanced designs combine client‑side memory with remote caches and use consistent hashing to minimize data movement when nodes are added or removed.

Storage Technology (NoSQL)

NoSQL databases (e.g., MongoDB, Redis) are favored for high‑concurrency services because they store simple key‑value or document data without complex relational schemas, allowing horizontal scaling and easy sharding based on a primary index.

Management Challenges of Distributed Systems

Hardware Failure Rate

With dozens or hundreds of servers, hardware failures become inevitable. Systems must detect and tolerate node failures without losing service, often by replicating data and providing automatic failover.

Resource Utilization Optimization

Adding hardware does not linearly increase capacity; coordination overhead can limit scalability. Expanding a cluster often requires stopping services, reconfiguring, and restarting, especially for stateful services like online games.

Software Service Updates

Batch deployment tools are essential for updating thousands of servers. Beyond copying binaries, deployment may involve firewall changes, shared memory setup, database schema migrations, and installing new software.

Version upgrades also raise data‑format compatibility issues; designing flexible data structures or version‑compatible protocols helps mitigate these problems.

Data Statistics and Decision Making

Log aggregation at scale requires distributed processing frameworks such as Google’s MapReduce. While powerful, these frameworks differ from traditional SQL, so many teams still offload aggregated results to relational databases for further analysis.

Overall, mastering distributed system techniques—layered architecture, concurrency models, caching, NoSQL storage, fault tolerance, scaling, deployment, and monitoring—is crucial for backend engineers building high‑performance, reliable internet services.