Scaling Vivo Cloud Service Data Storage: Sharding, Partitioning, and InnoDB Compression Strategies

Vivo Cloud Service offers backup for contacts, SMS, notes, bookmarks and other user data, with MySQL as the underlying storage engine.

As the service grew, user count jumped from millions to tens of millions and stored data increased from hundreds of billions to trillions of rows, creating a massive storage challenge.

Four sharding strategies were applied:

1. Horizontal partitioning (horizontal sharding) – Single tables such as browser bookmarks and notes, each exceeding 100 million rows, were split into 100 tables, each holding about 10 million rows.

2. Horizontal database sharding (horizontal scaling) – The original single database was split into 10 databases, and the 50 tables per database were expanded to 100 tables, migrating billions of rows.

3. Vertical sharding (vertical partitioning) – Storage analysis showed a single database used 5 TB, with contacts occupying 2.75 TB (55 %) and SMS 1 TB (20 %). The remaining space was insufficient for future growth.

4. Dynamic scaling via routing tables – A user‑router table records which database and table each user's contacts reside in. New users are routed to newly added databases, while existing users stay in their original shards, preserving space for legacy data.

Data compression research

Three options were evaluated:

Application‑level compression before storing – no DB changes but high CPU cost for massive existing data.

MySQL InnoDB built‑in compression – simple to enable, suitable for read‑heavy workloads.

TokuDB engine – strong compression but requires extra plugin and operational risk.

The team selected InnoDB compression for its low migration cost and adequate performance.

Configuration steps (wrapped in code tags):

SET GLOBAL innodb_file_format=Barracuda;
SET GLOBAL innodb_file_format_max=Barracuda;
SET GLOBAL innodb_file_per_table=1;

Enabling innodb_file_per_table is required because the system tablespace cannot be compressed.

Testing showed a 50 % compression ratio (e.g., a 20 MB table reduced to 10 MB) and acceptable CPU impact (CPU rose from ~33 % to ~43 % during heavy inserts, while TPS remained stable).

Both offline and online validations confirmed that compressed tables function correctly and do not degrade user‑visible performance.

Final online results demonstrated that contact storage space dropped from 65 % to 33 % of the allocated capacity, meeting the target of keeping at least 60 % free space for future growth.

Conclusion: Proper sharding combined with InnoDB compression can effectively handle massive data growth in cloud services, provided thorough pre‑deployment testing and careful monitoring of CPU usage.