MongoDB Sharding Key Query Failure and Unique Index Violation Explained

A MongoDB development team reported that a sharded collection could not be queried using the shard key, while queries by _id worked, and that a unique compound index seemed to allow duplicate documents. The article reproduces the problem and analyses its cause.

Environment

MongoDB version: mongos 4.0.12 (community)
Database: mdb_test
Sharded collection: mdb_test_coll
Shard key: key3 (hashed)

Index definition

db.mdb_test_coll.getIndexes()
[
  { "v": 2, "key": { "_id": 1 }, "name": "_id_", "ns": "mdb_test.mdb_test_coll" },
  { "v": 2, "key": { "key3": "hashed" }, "name": "key3_hashed", "ns": "mdb_test.mdb_test_coll" },
  { "v": 2, "key": { "key2": 1 }, "name": "key2_1", "background": true, "ns": "mdb_test.mdb_test_coll" },
  { "v": 2, "unique": true, "key": { "key3": 1, "key1": 1, "key2": 1, "key4": 1 }, "name": "key3_1_key1_1_key2_1_key4_1", "background": true, "ns": "mdb_test.mdb_test_coll" }
]

Data simulation

{ "_id": ObjectId("606ed113137fa535ac53d3ee"), "key1": null, "key2": 1033072826, "key3": BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="), "key4": "hls/.../hl" }
{ "_id": ObjectId("5fb2df3b32b0f42dced04ea7"), "key1": null, "key2": 2100000000, "key3": BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="), "key4": "hls/.../hl" }

Reproducing the issue

mongos> db.mdb_test_coll.save({ ... })   // insert first document
mongos> db.mdb_test_coll.find({"_id": ObjectId("606ed113137fa535ac53d3ee")})   // returns document
mongos> db.mdb_test_coll.find({"key3": BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="), "key2": 1033072826})   // no result

Inserting two documents with identical values for the unique compound index fields ( key3 , key1 , key2 , key4 ) does not raise a duplicate‑key error, confirming that the unique constraint is not enforced.

Root cause analysis

The server parameter failIndexKeyTooLong controls the behavior when an index entry exceeds the Index Key Length Limit (1024 bytes). When set to true (default), MongoDB aborts the operation with an error. When set to false , oversized entries are stored in the collection but omitted from the index, so the index does not enforce uniqueness and queries that rely on it cannot locate the documents.

db.adminCommand({ setParameter: 1, failIndexKeyTooLong: false })

Because the hashed shard key key3 and the compound unique index contain large values (binary data and long strings), their index entries exceed 1024 bytes. With failIndexKeyTooLong: false , the entries are not indexed, leading to the observed query failures and duplicate‑key acceptance.

Practical recommendations

When encountering similar symptoms, force the use of a different index (e.g., query by _id or a single field whose index entry is small).

Review schema design to avoid excessively large indexed fields; consider trimming or hashing values before indexing.

DBAs should discuss with developers whether to keep failIndexKeyTooLong enabled (recommended) to prevent silent index loss.

In summary, the issue is not a bug in MongoDB but a configuration effect: oversized index entries are silently dropped from the index when failIndexKeyTooLong is disabled, causing both query failures and apparent unique‑index violations.