Comparison of Kuaishou BlobStore and 58 WOS Object Storage Systems

On December 24, 2018, the 58 Group Technology Salon (Session 6) was held at the Beijing headquarters, featuring presentations by Kuaishou’s big‑data architecture team and 58’s storage R&D team on their respective object storage systems.

1. Kuaishou Object Storage System – BlobStore

BlobStore supports Kuaishou’s short‑video app, providing high availability, scalability, performance, and reliability for billions of video files. It is built on open‑source HBase and HDFS, combining offline storage techniques into an online service.

1.1 Overall Architecture

BlobStore uses a Gateway layer (HTTP server and business logic), a Memcache caching layer, an HBase index layer for metadata, and an HDFS data layer where video objects are merged into large files.

1.2 High Scalability

The system scales horizontally: stateless Gateways can be added arbitrarily, HBase can expand by adding nodes, and HDFS clusters are expanded as whole units. Data is stored across two data centers with asynchronous replication for metadata and dual‑write for object data.

1.3 High Availability

BlobStore employs a Hawk module (master + workers) to monitor node health and quickly remove failed nodes. Gateway and storage node failures trigger fast failover and circuit‑breaker mechanisms, with unhealthy nodes demoted and eventually removed.

2. 58 Object Storage System – WOS

WOS is a self‑developed object storage service for 58 Group, handling videos, audio, images, and documents at massive scale.

2.1 Overall Architecture

Based on Facebook’s "Finding a Needle in Haystack" paper, WOS merges small files into large ones. Its layers include Proxy (HTTP server, authentication, permission), Directory (metadata stored in a custom KV store wtable and scheduling), Store (actual disk storage), and Detector (node health monitoring).

2.2 "Instant Upload" Feature

To avoid duplicate uploads, WOS checks fragment hashes on the client side and uses reference counting on the server; identical fragments are not stored again, reducing bandwidth and storage.

2.3 Erasure‑Coding Storage

Initially using triple replication, WOS added a 4+2 erasure‑coding scheme, storing six encoded fragments per object, improving storage efficiency by about 50% without sacrificing data safety.

3. Summary and Comparison

Both BlobStore and WOS share similar layered architectures (access, index, storage, detection). BlobStore includes a caching layer due to hotspot video access, while WOS currently does not. BlobStore relies on HBase/HDFS, whereas WOS uses a custom KV store (wtable). BlobStore uses dual‑write across data centers; WOS combines triple replication with erasure coding. Both systems are critical to their companies and can learn from each other's designs.