Weibo's Third‑Generation Backend Architecture: Layers, Frameworks, and Monitoring
The article describes how Weibo evolved from a LAMP‑based first generation to a third‑generation, orthogonal‑decomposed backend architecture featuring three horizontal layers (interface, service, resource) and vertical technical components such as Web V4, MCQ queue, Motan RPC, object storage, SSD cache, and the WatchMan distributed tracing system for monitoring and service governance.
In 2014, Sina Weibo reported 143 million monthly active users and over 800,000 posts per minute, demanding a highly available, high‑concurrency, low‑latency backend. The platform progressed from a first‑generation LAMP stack (MyISAM, PHP, Memcache) to a second‑generation SOA architecture with Java services, and finally to a third‑generation system designed for rapid, reliable product delivery.
The third‑generation architecture uses an orthogonal decomposition method, forming a 3×4 matrix of zones. Horizontally it adopts a three‑tier model—interface layer, service layer, and resource layer—while vertically it separates business architecture, technical architecture, monitoring platform, and service‑governance platform.
Horizontal Layering
Interface layer: provides unified APIs for web pages and mobile clients, with core services for Feed, user relationships, and messaging.
Service layer: splits into atomic services (e.g., short‑link, ID generator) that have no dependencies, and composite services (e.g., Feed, messaging) that orchestrate multiple atomic services.
Resource layer: stores data models using caches (Redis, Memcache) and persistent stores (MySQL, HBase, distributed file system TFS, Sina S3).
Dependencies flow strictly from upper layers to lower layers, ensuring a simple, directed relationship.
Vertical Technical Architecture
The platform has built numerous middleware components driven by business needs, forming a comprehensive technical framework that boosts development efficiency and operational stability.
Interface Layer – Web V4 Framework
Built on Spring AOP and Jersey, the framework standardizes API development, offering built‑in authentication, rate limiting, logging, degradation, and automatic Bean‑JSON/XML serialization.
Service Layer Frameworks
MCQ Message Queue : A SimpleQueue Service over Memcache that persists messages in BerkeleyDB, supports only get/set commands, and provides easy monitoring via stats queue. It accelerates front‑end response time and enables asynchronous DB writes and real‑time data feeds to search, big‑data, and business analytics.
Motan RPC Framework : Uses Netty as the transport layer, supports Hessian and Java serialization, and offers multiple protocols (Motan, HTTP, TCP, MC). It provides high availability and load‑balancing strategies (FailOver, FailFast, Round‑Robin, LRU, Consistent Hash) and comprehensive service‑governance data such as call chains, QPS, latency, and standardized error logs.
Resource Layer Frameworks
Object Store : Serializes JVM objects into HBase with a unique ObjectID, supporting PB, JSON, and binary formats. It stores large media objects (videos, images, articles) and keeps download URLs in Sina S3.
SSDCache : Introduces an L2 cache layer (Redis/MC + SSD + MySQL) that leverages SSD’s high IOPS to reduce cost, alleviate DB pressure, and improve cache performance.
Monitoring and Service Governance – WatchMan Distributed Tracing System
Inspired by Google Dapper and Twitter Zipkin, WatchMan collects foot‑print logs from all middleware (interface, RPC, resource) with a low‑invasiveness principle. It reconstructs complete request call chains, gathers performance metrics, tracks errors, and feeds SLA data back to control flows for throttling, degradation, scaling, traffic shifting, and gray releases.
Overall, the technical framework now plays a crucial role in driving platform upgrades, business development, and operational services, with future articles planned to dive deeper into core middleware design principles.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Architecture Digest
Focusing on Java backend development, covering application architecture from top-tier internet companies (high availability, high performance, high stability), big data, machine learning, Java architecture, and other popular fields.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
