Weibo Database Architecture: Evolution, Optimization, and Design Practices

Author Background : Xiao Peng, Technical Manager at Weibo R&D Center, oversees MySQL, Redis, HBase, and Memcached operations, performance tuning, architecture design, and automation systems, with over ten years of internet database experience.

Database Expert Reflections : Xiao shares his journey from early interest in databases to becoming a DBA, emphasizing the importance of challenging environments and large‑scale platforms for skill growth.

Weibo Database Evolution

Initial Stage : Simple read/write‑separated architecture (1M/2S/1MB) with a master for writes and slaves for reads; vertical partitioning by business modules (user, content, relationship) using separate databases.

Vertical splitting provided clear module isolation and easy degradation, suitable for early growth without over‑design.

Explosion Stage : Rapid user growth required scale‑up with high‑performance hardware (PCIe‑Flash) and vertical sharding of modules (user, relationship, posts, etc.). MySQL write QPS peaked at 35,000 during the 2012 Spring Festival.

Horizontal sharding was applied to large tables such as posts: index and content were split, then each part was hashed and further divided by time, keeping table sizes manageable.

Index‑content split to store smaller index tables separately.

Hash‑based horizontal partitioning followed by time‑based sub‑partitioning.

Use of Memcached to cache content IDs after index lookup.

Consolidation Stage : Automation became the focus. Systems built include backup, monitoring, AutoDDL, MHA, inspection, slow‑query analysis, and the iDB portal for users to view DB status and submit DDL changes for automated execution.

Standardized MySQL development guidelines (e.g., quoting all WHERE literals) were introduced to prevent silent conversion errors and reduce runtime failures.

Recent Optimizations and Design

The platform now runs MySQL, Redis, Memcached, and HBase. Redis received several enhancements:

POS‑based replication : Modified AOF to record sync position, reducing bandwidth spikes during partial resynchronization.

Online hot upgrade : Dynamic loading of libredis.so enables version changes without master switch‑overs.

Custom extensions : Developed redisscounter (array‑based storage) and a Bloom‑filter‑based phantom for specific use cases.

Redis Middleware – Tribe : A proxy‑based system with a central config server, slot‑based sharding (inspired by Redis Cluster), automatic migration, fail‑over, and exposed APIs for integration with other automation tools.

Databus : An open‑source service that reads MySQL row‑based binlogs, applies custom business logic (via loaded .so modules), and synchronizes data to heterogeneous stores such as Redis and HBase. Future work includes Redis‑to‑MySQL sync.

Design practices for large‑scale tables include pre‑splitting, using message queues for eventual consistency, and adding an API layer to simplify application code.

Future Plans

Weibo aims to adopt additional databases (PostgreSQL, SSDB), leverage MySQL 5.7 features (parallel replication, GTID, dynamic BP), service‑ify NoSQL stores via proxies, and use cloud resources for dynamic cache scaling.

Selected Q&A Highlights

1. Index‑content separation : Implemented at the business layer; middleware handles horizontal sharding, while vertical split keeps index and content in different DB instances.

2. Notable outage : A mistaken DROP TABLE was recovered via rapid single‑table restore, prompting stricter drop‑table procedures (rename‑then‑wait‑24‑hours).

3. Hash‑based sharding : Uses modulo 1024 on post IDs, creating ~1024 tables per month; time‑based tables enable easy archiving.

4. Databus implementation : Parses MySQL binlog rows, loads custom logic via .so files, and streams changes to downstream stores.

5. Index vs. content : Index stores IDs and metadata; content is large and stored separately to keep query results small.

6. Role of NoSQL : Redis counters replace MySQL for high‑concurrency increments, dramatically improving TPS.

7. IoT data storage : Suggests HBase for massive time‑series data; MySQL JSON may help but has scalability limits.

8. Data‑index separation : Refers to vertical split across different DB instances, not merely file‑level separation.

9. Data placement : MySQL holds persistent data; Redis/Memcached serve hot‑spot reads; specialized services (e.g., rediscounter) live solely in NoSQL.

10. Global ID generators : Custom Redis‑based generator offers performance and encoded metadata versus MySQL auto‑increment IDs.