Designing Scalable High‑Availability Architecture for Hundreds of Millions of Users

The core of high‑availability architecture is decoupling and redundancy, including stateless design, database partitioning, caching, CDN, and DNS strategies; selecting the right technologies at each product‑iteration stage balances cost and future growth.

Starting with a single‑server monolith (web server + database) quickly reveals failure points: a database or web server crash brings the whole system down, and there is no fault‑tolerance or redundancy.

Introducing DNS resolution separates client address lookup from the service IP, allowing clients to discover the server address before sending requests.

Scalability Techniques

Scaling up (vertical) – add CPU, memory, storage, or network bandwidth to an existing server.

Scaling out (horizontal) – add more servers or instances, requiring load balancing and code changes for parallel processing.

Load balancers (hardware or software such as HAProxy, Nginx) distribute traffic across multiple back‑end servers using algorithms like round‑robin, least connections, fastest response, weighted, and IP‑hash.

Database Scalability

Replication (master‑slave, master‑master) for redundancy and read/write distribution.

Federation – split databases by functional domains.

Sharding – partition data horizontally by key (e.g., user ID) so each shard holds a subset of rows.

Denormalization – duplicate data across tables to reduce expensive joins and improve read performance.

SQL tuning and materialized views for further optimization.

Choosing between SQL (MySQL, PostgreSQL, Oracle, etc.) and NoSQL (key‑value, document, wide‑column, graph, blob) depends on data model, query patterns, and scalability needs; many enterprises use both.

To achieve stateless web layers, move session state to a shared store (SQL or NoSQL) so any web server can handle any request, enabling horizontal scaling.

Advanced Concepts

Cache – place frequently accessed data in memory (e.g., Redis) to reduce latency and load.

CDN – cache static assets at edge locations to lower page load time and increase availability.

GeoDNS – resolve domain names to the nearest data‑center IP, supporting global user bases.

Applying all these techniques iteratively—stateless architecture, load balancers, caching, multi‑data‑center deployment, CDN, sharding, etc.—allows a system to scale beyond 100 million users while maintaining reliability and performance.

Additional considerations include combining sharding with backup, choosing between long‑polling, WebSockets, or server‑sent events, indexing, proxying, SQL tuning, and elastic compute resources.