Why Is Your Elasticsearch Cluster Yellow? 5 Causes & Fixes for Unassigned Replicas

Elasticsearch yellow status means all primary shards are assigned but at least one replica shard is unassigned. The cluster remains readable and writable, but there is a single point of failure risk if a primary shard fails.

Step 1: Collect Core Status Information (Quick Problem Location)

1.1 Check Cluster Overall Health

Run the following command to get key metrics and avoid blind troubleshooting.

# curl way (replace with your ES address and port)
curl -XGET "http://localhost:9200/_cluster/health?pretty"
# Kibana Dev Tools way
GET /_cluster/health?pretty

Key output interpretation: status: cluster overall status (green/yellow/red) number_of_nodes: number of available nodes in cluster number_of_data_nodes: number of available data nodes unassigned_shards: total number of unassigned shards (yellow status means this value > 0)

1.2 Check Target Index Shard Allocation Details

Focus on the problematic index (replace your_index_name with actual index name):

# curl way
curl -XGET "http://localhost:9200/_cat/shards/your_index_name?v"
# Kibana Dev Tools way
GET /_cat/shards/your_index_name?v

Key output interpretation: shard: shard number (e.g., 0, 1, 2) prirep: shard type ( p = primary, r = replica) state: shard state ( STARTED = assigned, UNASSIGNED = unassigned) node: node hosting the shard (empty if unassigned)

1.3 Check Unassigned Shard Specific Reason (Critical!)

This command pinpoints the root cause of replica shard non-allocation:

# curl way
curl -XGET "http://localhost:9200/_cluster/allocation/explain?pretty"
# Kibana Dev Tools way
GET /_cluster/allocation/explain?pretty

Key output interpretation: index: index owning the unassigned shard shard: unassigned shard number primary: whether it is a primary shard (yellow status shows false, meaning replica unassigned) reason: core reason for non-allocation (e.g., disk_threshold_exceeded, no_valid_shard_copy)

Step 2: Analyze Yellow Status Common Causes and Fixes (Ordered by Probability)

Cause 1: Insufficient Nodes (Most Common!)

Phenomenon: Index replica count > available data nodes. ES never places a replica on the same node as its primary to avoid single point of failure. Example: 1 data node, replica set to 1 → replica has no node to allocate, index turns yellow.

Verification: Check index replica setting:

# curl way
curl -XGET "http://localhost:9200/your_index_name/_settings?pretty"
# Kibana Dev Tools way
GET /your_index_name/_settings?pretty

Output field number_of_replicas is the replica count; compare with cluster number_of_data_nodes.

Fix:

Temporary (quick green recovery): Reduce replica count to "available nodes - 1" (e.g., 1 node → set 0):

# curl way
curl -XPUT "http://localhost:9200/your_index_name/_settings" -H "Content-Type: application/json" -d '{
  "index": {
    "number_of_replicas": 0  # 1 node set 0, 2 nodes set 1, etc.
  }
}'
# Kibana Dev Tools way
PUT /your_index_name/_settings
{
  "index": {
    "number_of_replicas": 0
  }
}

Long-term: Add data nodes, then adjust replica count back to a reasonable value (e.g., 2 nodes → 1, 3 nodes → 1 or 2).

Cause 2: Disk Space Insufficient (ES Protection Mechanism)

Phenomenon: Node disk usage exceeds ES watermarks (default: ≥85% rejects shard allocation, ≥90% makes index read-only), preventing replica allocation.

Verification: Check node disk usage:

# curl way
curl -XGET "http://localhost:9200/_cat/nodes?v&h=name,ip,disk.used,disk.avail,disk.usage"
# Kibana Dev Tools way
GET /_cat/nodes?v&h=name,ip,disk.used,disk.avail,disk.usage

Field disk.usage shows disk usage percentage.

Fix:

Emergency cleanup: Delete unused indices, log files to free disk space (ensure usage < 85%).

Adjust disk watermarks (temporary): Modify elasticsearch.yml or dynamically adjust:

# Kibana way: set disk usage thresholds to 90%
PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.disk.watermark.low": "88%",
    "cluster.routing.allocation.disk.watermark.high": "89%",
    "cluster.routing.allocation.disk.watermark.flood_stage": "90%"
  }
}

Long-term: Expand node disks or add new data nodes.

Cause 3: Shard Allocation Strategy Disabled/Restricted

Phenomenon: Shard allocation manually disabled, or node filtering rules configured (e.g., index only allowed on nodes with specific label, but no matching nodes exist).

Verification:

Check if shard allocation is disabled: GET /_cluster/settings?pretty If cluster.routing.allocation.enable is none or primaries (only primaries allocated), replicas cannot be assigned.

Check node attribute filtering:

See if index specifies node attributes (e.g., node.attr.rack: rack1): GET /your_index_name/_settings?pretty If index.routing.allocation.require.* exists, confirm cluster has nodes with that attribute.

Fix:

Enable all shard allocation:

PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

Remove unreasonable node filtering rules (e.g., delete rack requirement if not needed):

PUT /your_index_name/_settings
{
  "index.routing.allocation.require.rack": null
}

Cause 4: Node Failure/Offline

Phenomenon: Node hosting replica shards crashes or loses network connectivity; ES hasn't yet reallocated replicas to other nodes.

Verification: Check node status: GET /_cat/nodes?v&h=name,status,ip If node status is down, its replica shards become UNASSIGNED.

Fix:

Restart failed node, wait for ES auto-recovery.

If node cannot recover, manually trigger shard reallocation:

# Trigger cluster rebalance shards
POST /_cluster/reroute?retry_failed=true

Cause 5: Shard Size/Count Exceeds Limits

Phenomenon: Single shard exceeds 50GB (ES best practice), or index total shard count too high, causing allocation timeout/failure.

Verification: Check index shard sizes:

GET /_cat/indices/your_index_name?v&h=index,docs.count,store.size

Fix:

Split large index: create new indices by time/business, keep shard size 10-50GB.

Adjust shard count (requires reindex, cannot change directly):

# Example: reindex with reasonable shard and replica count
POST /_reindex
{
  "source": {
    "index": "old_big_index"
  },
  "dest": {
    "index": "new_small_index",
    "settings": {
      "number_of_shards": 5,  # adjust per data volume
      "number_of_replicas": 1  # adjust per node count
    }
  }
}

Step 3: Verify Fix Results

After applying fixes, re-check cluster and index status to confirm green recovery:

# Check cluster health
GET /_cluster/health?pretty
# Check target index status
GET /_cat/shards/your_index_name?v

If status becomes green, and all shards ( p + r) have state = STARTED, the fix is successful.

Summary

Check node count and replica count first: Most common trigger; 1 node with replica 0 quickly restores green.

Then check disk space: Ensure node disk usage < 85% to avoid ES protection blocking allocation.

Finally check allocation strategy/node status: Disabled allocation and node failures are also frequent causes.

Fix principle: First temporarily restore green (ensure availability), then do long-term optimization (scale out, adjust shard config).