Master MongoDB Sharding: Theory, Setup, and Troubleshooting

Hello everyone, I'm Su San. In this article we explore why MongoDB is used for storing images and documents instead of MySQL, and how to set up a MongoDB sharding cluster.

Quick resolution of a MongoDB production incident

Understanding what MongoDB sharding is

How MongoDB performs sharding

When to apply sharding

Building a MongoDB sharding server

How MongoDB tracks data in a sharded cluster

MongoDB connection refused? The service has likely crashed.

Check whether the MongoDB process is running:

ps -aef|grep mongo

The most likely cause is a full disk.

Check disk space:

df -TH

If the disk is 100% full, delete log files and restart MongoDB: rm -rf * in the log directory, then restart the service.

MongoDB startup exception: "about to fork child process, waiting until server is ready for connection"

Because the cluster performs data synchronization on startup, the process may take time. If you interrupt with Ctrl+C, force‑stop the processes and restart.

List MongoDB processes:

ps -aef|grep mongo

Force‑kill all MongoDB processes:

ps -aef|grep mongo | grep -v grep | awk '{print $2}' | xargs kill -9

Delete mongod.lock and diagnostic.data in the data directory, then restart using the script mongos_start.sh (which runs mongod --config data/mongodb.conf).

1. What is MongoDB sharding?

Sharding is the process of distributing data across multiple machines, also known as partitioning.

MongoDB supports manual sharding, where the application maintains connections to multiple independent database servers. This approach becomes hard to maintain when nodes are added or removed or when data distribution changes.

MongoDB also supports automatic sharding, which abstracts the sharding logic away from the application and balances data across shards automatically, simplifying node addition and removal.

A sharded cluster consists of many shards; each shard holds a subset of the data. Applications connect to one or more mongos routing processes, which maintain a metadata directory indicating which shard contains which data. The application sees the cluster as a single logical server.

2. How does MongoDB perform sharding?

On a single machine you can quickly create a test cluster using the --nodb and --norc options: mongo --nodb --norc Use the ShardingTest class to create a cluster:

st = ShardingTest({
  name:"one-min-shards",
  chunkSize:1,
  shards:2,
  rs:{
    nodes:3,
    oplogSize:10
  },
  other:{
    enableBalancer:true
  }
});

name : label for the sharding cluster

shards : number of shards (2 in this example)

rs : each shard is a replica set of 3 nodes

enableBalancer : activates the balancer after startup ShardingTest creates a cluster with two shards, each a replica set, starts a mongos router, and launches a config‑server replica set to direct queries to the correct shard.

Sharding is typically used to overcome hardware and cost limits or to improve performance.

After ShardingTest runs, you have ten processes: two replica sets (3 nodes each), one config‑server replica set (3 nodes), and one mongos listening on port 20009.

3. When should you shard?

Sharding is appropriate when you need to:

Increase available RAM

Increase available disk space

Reduce server load

Handle throughput beyond a single MongoDB instance's capacity

4. Building a MongoDB sharding server

1) Config server (config process)

The config server stores metadata about which data resides on which shard. Start it before any mongos using mongod -f config.conf. It uses "majority" write and read concerns to ensure metadata consistency.

2) mongos process

The mongos router is started with mongod -f config.conf and must know the config server addresses via configdb=configReplSet/host1,host2,host3. Deploy multiple mongos instances close to all shards to improve query performance.

3) Converting a replica set to a shard

After the config server and router are running, add existing replica sets as shards. From MongoDB 3.4 onward, each shard's mongod must be started with the --shardsvr option (add shardsvr=true to config.conf on every member).

4) Data sharding

Example: shard the worker collection in the test database on the name field.

Enable sharding on the database: sh.enableSharding("test") Shard the collection: sh.shardCollection("test.worker", {"name":1}) If the collection already exists, an index on the shard key ( name) is required; otherwise MongoDB creates the index automatically.

The shardCollection command splits the collection into chunks that are evenly distributed across shards.

5. How does MongoDB track cluster data?

1) Data chunks

MongoDB groups documents into chunks based on the shard key range. A small internal collection maps chunks to shards.

Key rules:

Chunks must not overlap.

Overly large chunks are automatically split.

Each document belongs to exactly one chunk.

2) Chunk ranges

Newly sharded collections start with a single chunk covering the full key range (‑∞ to +∞).

As chunks grow, MongoDB splits them into smaller ranges, e.g., (‑∞, value) and (value, +∞).

This allows mongos to quickly locate the chunk containing a document.

3) Splitting chunks

Each shard’s primary mongod monitors its chunks. When a chunk reaches a size threshold, the shard requests the global chunk size from the config server, splits the chunk, and updates the metadata.

If a write targets a chunk that exceeds the threshold, the shard may request the balancer to migrate the top chunk to another shard.

Only documents with different shard‑key values can be split; identical shard‑key values stay in the same chunk.

{"name":"哪吒编程","book":"Java核心技术","readTime":"October"}
{"name":"哪吒编程","book":"Java编程思想","readTime":"October"}
{"name":"哪吒编程","book":"深入理解Java虚拟机","readTime":"October"}
{"name":"哪吒编程","book":"effective java","readTime":"November"}
{"name":"哪吒编程","book":"重构 改善既有代码的设计","readTime":"November"}
{"name":"哪吒编程","book":"高性能MySQL","readTime":"December"}
{"name":"哪吒编程","book":"Spring技术内幕","readTime":"December"}
{"name":"哪吒编程","book":"重学Java设计模式","readTime":"December"}
{"name":"哪吒编程","book":"深入理解高并发编程","readTime":"January"}
{"name":"哪吒编程","book":"Redis设计与实现","readTime":"January"}

If a chunk cannot be split because the shard key values are identical, MongoDB may experience a "split storm" where the shard repeatedly attempts and fails to split.

6. The balancer

The balancer runs on the primary member of the config‑server replica set. It periodically checks for uneven chunk distribution and migrates chunks when a shard exceeds a migration threshold.

Only when a shard’s chunk count reaches the configured threshold does the balancer become active, ensuring balanced data distribution across the cluster.