How to Architect Systems for 100M Users: Strategies, Tech Stack, and Cost Tips

What Does 100M Users Mean?

When a service reaches 100 million registered users, typical metrics include 20‑30 million daily active users, peak QPS of 500k‑1M, storage requirements in the terabyte range, and bandwidth needs of tens of Gbps.

These numbers create three core challenges: performance bottlenecks , availability requirements , and data consistency . Any weakness can cause system collapse.

Layered Architecture Design Strategy

Access Layer: Traffic Distribution

Smart DNS (GeoDNS) routes users to the nearest node, reducing latency by 30‑50%.

CDN + Edge Computing serve static assets via CDN and pre‑process dynamic content at edge nodes, handling 70‑80% of requests close to users.

Load‑Balancing Strategy follows a multi‑layer chain: Internet → DNS → CDN → L4 load balancer → L7 load balancer → application cluster. L4 uses LVS or F5; L7 uses Nginx or Envoy, enabling million‑level QPS.

Application Layer: Micro‑service Partitioning

Monoliths cannot meet 100M‑user scale; micro‑services must be split wisely.

Service Splitting Principles :

Domain‑driven design (DDD)

Team size considerations (Conway's Law)

Avoid distributed transactions

Maintain low coupling between services

Typical core services: user, content, recommendation, payment, notification, each deployed independently behind an API gateway.

Service Governance includes service discovery (Consul/Eureka), configuration management (Apollo/Nacos), circuit breaking (Hystrix/Sentinel), and tracing (Jaeger/SkyWalking).

Data Layer: Storage Design

Balancing consistency, availability, and partition tolerance is essential.

Cache Hierarchy : Application → Local cache (Caffeine) → Distributed cache (Redis Cluster) → Database.

Database Sharding :

Vertical split by business module

Horizontal split by user ID or time

Example: user data across 256 shards, content sharded by content ID, logs sharded by date

Read‑Write Separation & Multi‑Active : Primary handles writes, replicas serve reads; multi‑active zones serve regional users to avoid cross‑region latency.

Key Technology Selection and Implementation

Message Queue: Asynchronous Backbone

100M users generate massive event streams; a single Kafka cluster can process millions of messages per second.

Technology Choices :

High‑throughput: Apache Kafka

Low‑latency: Apache Pulsar

Transactional support: RocketMQ

Partition Strategy : Partition by user ID or business type, typically 32‑64 partitions to preserve ordering.

Search & Recommendation: Personalization Challenges

Search Architecture uses an Elasticsearch cluster with time‑ and region‑based shards; hot data on SSD, cold data on HDD.

Recommendation System combines offline batch processing (Spark) with online real‑time feature extraction and model inference, plus A/B testing for new algorithms.

Monitoring & Operations: Observability

Monitoring Stack covers:

Infrastructure: CPU, memory, network, disk

Application: QPS, latency, error rate

Business: user behavior, conversion rates

Alert Levels :

P0 – core function outage, respond within 5 minutes

P1 – partial outage, respond within 30 minutes

P2 – performance degradation, respond within 2 hours

Performance Optimization Practices

Database Optimization

Index Strategy :

Primary key: auto‑increment or Snowflake ID

Composite indexes follow left‑most prefix rule

Avoid excessive indexes on large tables

SQL Tuning :

Avoid SELECT * Use LIMIT to bound result sets

Batch operations where possible

Cache Optimization

Cache Penetration : Use Bloom filters to block invalid requests.

Cache Avalanche : Assign random expiration times.

Cache Breakdown : Keep hot keys permanent and refresh asynchronously.

JVM Tuning

-Xms8g -Xmx8g
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-XX:+HeapDumpOnOutOfMemoryError

Disaster Recovery & High Availability

Multi‑Level DR

Active‑Active in the Same City : Two data centers mirror each other in real time.

Active‑Active Across Regions : Independent deployments per region.

Hybrid Cloud Backup : Critical data backed up to the cloud.

Failure Drills

Single‑machine failure

Data‑center power outage

Network partition

Database master‑slave switch

Cost Control Strategies

Elastic Resource Scaling based on CPU/memory metrics, scheduled scaling during low‑traffic periods, and containerization for higher utilization.

Storage Cost Optimization :

Hot data on SSD, warm data on SATA, cold data on object storage

Compression and deduplication

Regular cleanup of expired data

According to AWS cost analysis, proper architectural optimization can reduce infrastructure spend by 30‑40%.

Technical Evolution Roadmap

Four phases guide growth:

Phase 1 (0‑1M): Monolith + read/write split

Phase 2 (1‑10M): Micro‑services + cache cluster

Phase 3 (10‑100M): Distributed architecture + multi‑active deployment

Phase 4 (100M+): Cloud‑native + intelligent operations

Each stage solves the dominant challenges while avoiding over‑design.

Conclusion

Designing for 100 million users is a systems engineering effort that balances performance, availability, consistency, and cost. Success relies on layered design, sensible service partitioning, appropriate technology selection, and continuous optimization driven by observability data.