Designing a System to Scale to 100 Million Users: Architecture, Scalability, and Database Strategies

Designing a system that can support tens of billions of users is a major challenge for software architects; this guide walks through practical steps to make scaling easier.

1. Start from a single server – Deploy a web application (e.g., on Apache or Tomcat) and a database (Oracle or MySQL) on the same machine. This simple setup lacks redundancy: if the database or web server fails, the whole system goes down.

2. The art of scalability – Two scaling directions are discussed: scale up (vertical scaling) and scale out (horizontal scaling). Vertical scaling adds CPU, memory, disks, or network bandwidth to an existing server but has limits such as hardware cost, OS constraints, and required downtime. Horizontal scaling separates web and database layers onto multiple machines, allowing independent optimization and fault isolation.

3. Use a load balancer – A load balancer (hardware or software such as HAProxy or Nginx) sits between clients and servers, distributing traffic via algorithms like round‑robin, least connections, fastest response, weighted, or IP hash. It improves availability and enables seamless addition or removal of servers.

4. Expand relational databases – Techniques include master‑slave replication, master‑master replication, sharding, denormalization, and SQL tuning. Replication keeps copies of data on multiple machines; sharding splits data across servers; denormalization trades write performance for faster reads.

5. Which database to use? – Discusses the trade‑offs between SQL (MySQL, Oracle, PostgreSQL, etc.) and NoSQL (key‑value, document, column, graph, blob). Highlights key‑value stores (Redis, Dynamo) and document stores (MongoDB, CouchDB) with inline code examples: key and value .

6. Horizontally scale the web layer – Move session state out of the web tier into a shared database or cache, creating a stateless architecture that works well with load balancers.

7. Advanced concepts – Caching reduces load on databases and web servers; CDNs store static assets close to users; GeoDNS routes users to the nearest data center, enabling global deployment.

8. Future topics – Mentions upcoming discussions on combining sharding with replication, long‑polling vs. WebSockets vs. Server‑Sent Events, indexing, proxying, SQL optimization, and elastic computing.

The article concludes that by iteratively applying these techniques, a system can be scaled to over 100 million users while remaining maintainable and cost‑effective.