Scaling from Zero to One Million Users: System Design Fundamentals

Designing a system for millions of users is an iterative process that starts with a single‑server prototype and progressively adds components to improve availability, performance, and scalability.

Single‑server configuration : All components (web application, database, cache) run on one machine; the request flow starts with DNS resolution, followed by the client contacting the server directly.

Database choices : Both relational (MySQL, PostgreSQL, Oracle) and NoSQL (Cassandra, DynamoDB, Redis) options are discussed, with guidance on when to prefer each based on latency, data structure, and volume requirements.

Vertical vs. horizontal scaling : Vertical scaling (adding CPU/RAM) is simple but limited and introduces single‑point‑of‑failure risks; horizontal scaling adds more servers to a pool, enabling load distribution and redundancy.

Load balancer : A load balancer distributes incoming traffic across multiple web servers, hides server IPs behind a public address, and provides failover by routing traffic to healthy instances.

Database replication : Master‑slave replication separates read and write workloads, improves performance, and provides high availability; failover procedures are described for both master and slave outages.

Caching layer : Introducing a cache (e.g., Memcached) reduces database load by storing frequently accessed data; common strategies such as read‑through caching, expiration policies, consistency considerations, and eviction algorithms (LRU, LFU, FIFO) are covered.

GET /users/12 – 获取 id=12 的用户对象</code>
<code>{</code>
<code>    "id": 12,</code>
<code>    "firstName": "John",</code>
<code>    "lastName": "Smith",</code>
<code>    "address":{</code>
<code>        "streetAddress": "21 2nd Street",</code>
<code>        "city": "New York",</code>
<code>        "state": "NY",</code>
<code>        "postalCode": 10021</code>
<code>    },</code>
<code>    "phoneNumbers": [</code>
<code>        "212 555-1234",</code>
<code>        "646555-4567"</code>
<code>    ]</code>
<code>}

Content Delivery Network (CDN) : Static assets (images, CSS, JavaScript) are offloaded to geographically distributed edge servers, reducing latency and bandwidth consumption; best‑practice tips on TTL, cost, and fallback handling are provided.

Stateless network layer : Session data is moved out of web servers into a shared persistent store (relational DB or NoSQL) so any server can handle any request, enabling easier horizontal scaling.

Multi‑data‑center deployment : Geo‑DNS directs users to the nearest data center; replication and failover strategies ensure continuity when a region experiences outages.

Message queue : Introducing a durable queue decouples producers and consumers, allowing asynchronous processing and independent scaling of workers.

Logging, metrics, and automation : Centralized logging, collection of host‑level, aggregate, and business metrics, and CI/CD pipelines are essential for operating large‑scale systems.

SECONDS=1</code>
<code>cache.set('myKey', 'hi there', 3600*SECONDS)</code>
<code>cache.get('myKey')

Database scaling : Vertical scaling adds resources to a single node, while horizontal scaling (sharding) distributes data across multiple nodes using a sharding key (e.g., user_id % 4). Considerations include choosing an even‑distribution key, handling hot‑spot keys, and dealing with cross‑shard joins via denormalization.

Checklist for supporting >1 000 000 users : make the network layer stateless, ensure redundancy at every layer, maximize caching, deploy multiple data centers, use a CDN for static content, shard the data layer, split architectural layers into separate services, and continuously monitor and automate operations.