Evolution of 58.com Architecture: From Single‑Server to Scalable Service‑Oriented System
The article chronicles how 58.com’s website architecture evolved through multiple growth stages—from a tiny single‑machine setup to a distributed, service‑oriented platform—by addressing scaling bottlenecks with technologies such as LAMP, read/write separation, vertical splitting, caching, and automated deployment.
For many startups it is difficult to predict what the website architecture will look like after traffic grows tenfold, hundredfold, or thousandfold, and building a massive concurrent architecture from the start is often prohibitively costly.
This article focuses on the evolution of architecture at each stage, identifying the problems faced and continuously solving them, allowing the architecture to evolve organically.
In the early days of 58.com the traffic was only around a hundred thousand visits, meaning only a few requests per second. The site was simple, with low request volume, small data size, and minimal code, easily handled by a few engineers, so there was essentially no dedicated architecture.
Initially the architecture can be described as “ALL IN ONE”: everything—site, database, files—was deployed on a single machine, and developers spent most of their time writing CRUD code that passed data from the front‑end to the business layer, then to the database, and finally back to the browser.
If the architecture were rebuilt today, 58.com would choose a LAMP stack because it requires no compilation, enables rapid feature delivery, and consists of mature, free open‑source components; a simple forum can be built in two days, making Windows unnecessary for early‑stage startups.
When traffic crossed the hundred‑thousand threshold, the database became a bottleneck and the system moved to a distributed architecture with multiple machines. Common techniques such as static/dynamic separation, read‑write separation, and master‑slave replication were introduced to handle the increased read‑heavy workload.
To decouple tightly coupled site components, 58.com vertically split business functions (e.g., home page vs. posting page) and partitioned large data sets into smaller databases, alleviating read‑write latency.
Static resources were served via CDN, MVC was adopted for clearer separation of concerns, and load‑balancing technologies were employed to improve performance and reliability.
As traffic grew to tens of millions, the original Windows‑based stack could no longer meet performance and cost requirements, prompting a migration to a Java‑based ecosystem covering operating system, database, and other layers.
During this high‑traffic phase, the number of services and machines expanded dramatically, leading to the creation of an independent service layer that unified request handling, introduced caching, and used reverse‑proxy, DNS clustering, and LVS to ensure high availability across the presentation, service, and data tiers.
58.com also built its own image storage system and open‑sourced its site and service frameworks (Argo and Gaea), allowing rapid configuration‑driven deployment.
When the architecture became a “spider web,” manual operations were no longer feasible; a configuration center was introduced to automate service discovery, scaling, and decommissioning, while flexible services automatically added instances under load.
Database access was centralized in the service layer, and data sharding was applied per business line to manage growing data volumes.
Automation now covers regression testing, operations, monitoring, and intelligent features such as recommendation, advertising, and search, all driven by underlying technology.
Looking ahead, 58.com faces challenges of wireless/mobile demand, rapid iteration, real‑time computation, and scaling the server fleet from roughly 3,000 to 10,000 machines, requiring further advances in parallel processing, automation, and operational efficiency.
In summary, different traffic stages present distinct problems; early stages prioritize development efficiency with ORM/DAO, medium stages adopt static/dynamic separation, read‑write splitting, and CDN, large‑scale stages introduce vertical splitting, service‑orientation, and reverse‑proxy, and billion‑level traffic demands centralized configuration, flexible services, message buses, and extensive automation.
Art of Distributed System Architecture Design
Introductions to large-scale distributed system architectures; insights and knowledge sharing on large-scale internet system architecture; front-end web architecture overviews; practical tips and experiences with PHP, JavaScript, Erlang, C/C++ and other languages in large-scale internet system development.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.