Scalable Architecture Without Magic: Principles and Practical Setups

Basic Principles

Choose Appropriate Tools

Different programming languages and databases excel at different tasks; for example, Python offers expressive syntax but runs on an interpreter and is slower than compiled Go or C, while Node.js is single‑threaded and requires a process manager like PM2 to stay stateless on multi‑core machines.

SQL provides Turing‑complete querying but is generally slower than NoSQL because it lacks caching; databases are often read‑oriented or write‑oriented, so selecting a write‑optimized store such as Cassandra for heavy writes or a read‑optimized store like MongoDB for news feeds is advisable.

Multiple Servers

Scaling out from one server to two is harder than adding subsequent servers; backend services must be stateless, storing all state in databases so that any server can handle any request, and load balancers (PM2 for Node.js or Nginx for others) should route traffic uniformly.

Caching and Rate Limiting

Introduce a caching layer (e.g., Varnish) so only the first request triggers a database query while subsequent requests are served from RAM; configure appropriate TTLs and never cache user input.

Implement a rate limiter to reject requests that arrive too quickly, protecting the server from a Slashdot effect.

Dividing Responsibilities

Leverage the database to compute foreign keys and other data‑intensive operations instead of the backend; move data‑related logic into stored procedures or queries.

Large Data Volumes

Even with clustered databases, capacity is limited by the total storage of all servers; to grow indefinitely, adopt distributed databases that spread data across many nodes, and add more servers as needed.

Potential Bottlenecks

Single‑threaded, stateful servers that cannot be load‑balanced.

Backend performing database work instead of delegating to the DB.

Single database instance without clustering.

Confusing read‑optimal and write‑optimal databases.

Geographic distance to users; use a CDN.

Setup Examples

Kitten (LAMP stack)

A simple, stateful LAMP setup suitable for small weekend projects; limited scalability due to in‑memory session storage.

Big Cat

Adds caching (MongoDB, Express) but remains stateful, so scalability is still constrained as user count grows.

Cheetah

Stateless architecture with Go, Redis cache, and MongoDB; can scale to many servers but still relies on a single database instance.

Tiger

Fully scalable with Go, Redis, Cassandra, and MongoDB clusters; handles millions of users but may suffer latency for distant users.

Lion (CDN)

Deploys multiple edge servers worldwide serving as full replicas of the origin; still limited by the capacity of the primary database.

Gear Tiger

Ultimate setup using a graph database like Riak for virtually unlimited storage; cost becomes the primary constraint.

Conclusion

Choose the right tool for each job, keep services stateless, use appropriate caching, rate limiting, and database delegation, and design the architecture to match the expected data volume and user distribution.

Guarantee scalability and statelessness!

http://pm2.keymetrics.io/?fileGuid=gr8wsimng4sTPe0C

https://varnish-cache.org/?fileGuid=gr8wsimng4sTPe0C