Why MongoDB’s Unique Index Fails When Index Keys Exceed 1024 Bytes

Problem Simulation

A development team reported that a sharded MongoDB collection could not be queried by its shard key, yet queries by _id succeeded, and a compound unique index appeared to allow duplicate records. The article reproduces the situation using MongoDB 4.0.12 (mongos), database mdb_test , collection mdb_test_coll , and shard key key3 .

Index information:

<code>db.mdb_test_coll.getIndexes()
[
  {
    "v" : 2,
    "key" : { "_id" : 1 },
    "name" : "_id_",
    "ns" : "mdb_test.mdb_test_coll"
  },
  {
    "v" : 2,
    "key" : { "key3" : "hashed" },
    "name" : "key3_hashed",
    "ns" : "mdb_test.mdb_test_coll"
  },
  {
    "v" : 2,
    "key" : { "key2" : 1 },
    "name" : "key2_1",
    "background" : true,
    "ns" : "mdb_test.mdb_test_coll"
  },
  {
    "v" : 2,
    "unique" : true,
    "key" : { "key3" : 1, "key1" : 1, "key2" : 1, "key4" : 1 },
    "name" : "key3_1_key1_1_key2_1_key4_1",
    "background" : true,
    "ns" : "mdb_test.mdb_test_coll"
  }
]
</code>

Sample documents (fields key1 is null, key2 is a large integer, key3 is binary data, key4 is a very long string) are inserted. Queries by _id return the document, while queries that include key3 or the compound unique index return no results.

Inserting two documents with identical values for the unique fields but different _id values does not raise a duplicate‑key error, and both records are stored.

Principle Analysis

The root cause is the server parameter failIndexKeyTooLong . When set to true (the default), any index entry larger than the 1024‑byte limit triggers an error. When set to false , oversized entries are allowed to be written but are omitted from the index, so the unique constraint is not enforced and queries that rely on that index cannot find the documents.

Changing the parameter at runtime:

<code>db.adminCommand({ setParameter: 1, failIndexKeyTooLong: false })
</code>

The 1024‑byte limit is documented as the "Index Key Length Limit". Screenshots (omitted here) illustrate the behavior when the parameter is true versus false.

Conclusion

For developers encountering this situation, explicitly forcing the use of a different index or adding a suitable index can work around the issue. Teams should evaluate whether such long index keys are necessary and discuss with DBAs whether failIndexKeyTooLong should remain enabled in production.