Evolution of 58.com Architecture from Small Traffic to Billion‑Scale

In the OneAPM technical open class, Shen Jian, executive chairman of the technical committee at 58.com, explains how the site’s architecture evolved as traffic grew from tens of thousands to over a billion visits.

Initially, the site ran on a single Windows/IIS/SQL‑Server/C# machine with minimal traffic, where developers mainly performed CRUD operations; the architecture was essentially an "ALL IN ONE" monolith.

Reflecting on that early choice, Shen suggests that if starting over, a LAMP stack would be preferable due to its open‑source nature, rapid deployment, and zero licensing cost.

When traffic crossed the 100k threshold, the database became a bottleneck. The team introduced distributed techniques such as static/dynamic separation, read‑write splitting with master‑slave replication, and horizontal scaling of servers.

To reduce coupling and latency, they vertically split business modules (e.g., homepage and posting page) and partitioned large data sets into smaller shards, also employing CDN for static assets and MVC for clearer responsibility separation.

As traffic exceeded ten million, performance and cost pressures forced a migration from the Windows stack to a Java‑based ecosystem, encompassing OS, database, and middleware changes.

Further growth required a service layer that centralized business logic, introduced caching to offload database reads, and used reverse proxy, DNS load balancing, and LVS to ensure high availability across site, service, and data layers.

The architecture later adopted flexible services that automatically scale with traffic, a configuration center for dynamic service discovery, and extensive automation for testing, deployment, monitoring, and operations.

Future challenges include supporting mobile and wireless access, real‑time computation for recommendations, and scaling the infrastructure from a few thousand to tens of thousands of servers.

Overall, the presentation highlights how different traffic stages demand distinct architectural solutions—starting with ORM/DAO for development efficiency, then moving to static/dynamic separation, read‑write splitting, CDN, MVC, and finally to service‑oriented, highly automated, and scalable designs for billion‑scale traffic.

Source: http://news.oneapm.com/shenjian-oneapm-course/