Facebook’s Shard Manager: Strategies for Large‑Scale System Sharding, Fault Tolerance, and Resource Utilization

Large‑scale system sharding must balance disaster recovery, failover, load balancing, and resource utilization. The article first distinguishes stateless services, which scale easily via traffic routing, from stateful services that require more sophisticated placement strategies such as consistent hashing and sharding.

Consistent hashing can cause data skew and is less friendly to multi‑data‑center deployments, whereas sharding offers finer‑grained control, supports multiple replicas (primary/secondary), and can map shards to servers based on user IDs, geographic location, and other policies.

In distributed environments, failures are treated as normal; high availability depends on replication, automatic detection, and controlled failover. Strategies include replication (master‑slave), automated health checks, failover throttling, and avoiding cascade failures that can collapse the entire service chain.

Resource efficiency is addressed through load balancing across heterogeneous hardware and dynamic resources, as well as elastic scaling that adjusts capacity according to time‑varying traffic patterns.

Facebook’s solution is the Shard Manager platform, built on a layered infrastructure: host management, container management, shard management, sharded applications, and products. The Shard Manager Scheduler collects load metrics, monitors server joins/failures, and drives shard state transitions via RPC calls, publishing a global shard view to the service‑discovery system for client routing.

The platform defines primitives for shard operations, such as status add_shard(shard_id) , status drop_shard(shard_id) , status change_role(shard_id, primary <-> secondary) , status update_membership(shard_id, [m1, m2, ...]) , and RPC client creation/execution. These primitives can be composed to implement higher‑level protocols, e.g., moving a shard from a heavily loaded server to a lighter one:

Status status = A.drop_shard(xx);
   if (status == success) {
       B.add_shard(xx);
   }

Shard Manager also supports configurable replication factors, failover delay policies, failover throttling, hardware‑aware load factors, multi‑resource balancing, and elastic scaling of shards according to traffic.

Reference: Facebook engineering article "Using Shard Manager to Scale Services".