How to Scale to 1 Billion Weekly Requests with HAProxy, PHP, Redis & MySQL

In this article the author demonstrates a simple yet powerful architecture that uses HAProxy, PHP, Redis, and MySQL to support roughly one billion requests per week, and outlines horizontal scaling paths and common patterns.

Background

About a year ago a fast‑growing e‑commerce startup needed a cost‑effective, scalable solution for international expansion. Their legacy LAMP stack required a minimal‑change approach, so the architecture had to stay simple enough for the existing PHP team to maintain long‑term.

The original system consisted of many independent front‑end sites, each with its own database and shared business‑logic code, leading to maintenance nightmares as code changes propagated across sites.

To address this, core business functions were extracted into independent services following a service‑oriented architecture and the “separation of concerns” principle. Front‑end sites now communicate via REST APIs (JSON), while less frequently used services such as authentication and session management are isolated.

Application Layer

Requests from front‑end sites first hit HAProxy, which load‑balances to three application nodes. Each node runs:

Xeon E5‑1620 @ 3.60 GHz, 64 GB RAM, SATA storage

Varnish cache

Apache 2

PHP 5.4.x (PHP‑FPM) with APC opcode cache

The nodes operate in an active‑active N+1 configuration, each with its own Varnish instance to avoid single points of failure. The application itself is built on the Symfony2 framework, a full‑stack PHP solution that provides a clear project structure, many ready‑made components, and excellent developer ergonomics.

Performance tests using JMeter showed that the Symfony2 application can handle 700 requests / s with an average response time of 50 ms, confirming that a complex framework does not necessarily impede throughput.

Monitoring is performed with Symfony2 tools, Monolog for logging (debug, error, critical levels), and external services such as Icinga and Collectd.

Data Layer

Redis

Redis serves as the primary data store, holding about 160 million records (~100 GB). It operates in a master‑slave configuration on two Xeon E5‑2650v2 @ 2.60 GHz, 128 GB RAM, SSD servers, with memory limited to 100 GB.

Key design choices:

O(1) or O(N) operations where N is the number of keys, ensuring keyspace size does not affect performance.

Bulk MGET of 100+ keys to minimize network latency.

Monitoring of key expiration is done via INFO keyspace and visualized with Icinga dashboards.

MySQL

MySQL is used as a tertiary cache layer, storing less‑frequently accessed objects to offload Redis. The setup includes two master servers (Xeon E5‑1620 @ 3.60 GHz, 64 GB RAM, SSD) with asynchronous master‑master replication and a separate slave for backup.

Vertical scaling is currently employed, but the data model is designed for easy sharding when horizontal scaling becomes necessary.

Lessons Learned

Do not share databases across services. Overloading Redis with session data can force caches onto MySQL, causing heavy load.

Log as much detail as possible. Rich log lines simplify debugging and reduce mean‑time‑to‑resolution.

Complex frameworks can still deliver high performance. Proper tooling and testing can make full‑stack solutions viable for high‑traffic workloads.