Elasticsearch Allocation Deciders: Overview, Configuration, and Core Algorithms

AllocationDecider is an abstract base class in Elasticsearch that enables dynamic shard allocation decisions at the node, cluster, or index level.

All concrete implementations can be listed with a grep command, for example:

➜  elasticsearch git:(7.5) rg -l --sort-files "extends AllocationDecider" server/src/main | sed 's/.*\///g'
AllocationDeciders.java
AwarenessAllocationDecider.java
ClusterRebalanceAllocationDecider.java
ConcurrentRebalanceAllocationDecider.java
DiskThresholdDecider.java
EnableAllocationDecider.java
FilterAllocationDecider.java
MaxRetryAllocationDecider.java
NodeVersionAllocationDecider.java
RebalanceOnlyWhenActiveAllocationDecider.java
ReplicaAfterPrimaryActiveAllocationDecider.java
ResizeAllocationDecider.java
RestoreInProgressAllocationDecider.java
SameShardAllocationDecider.java
ShardsLimitAllocationDecider.java
SnapshotInProgressAllocationDecider.java
ThrottlingAllocationDecider.java

Several key deciders are described:

AwarenessAllocationDecider : Uses node attributes (e.g., cluster.routing.allocation.aware.attribute: rack_id) to ensure replicas are placed on different racks or zones.

ClusterRebalanceAllocationDecider : Controls rebalancing based on shard activity; configured via cluster.routing.allocation.allow_rebalance with values such as indices_primaries_active, indices_all_active, or always.

ConcurrentRebalanceAllocationDecider : Limits the number of concurrent rebalancing operations using cluster.routing.allocation.cluster_concurrent_rebalance (default 2, -1 for unlimited).

DiskThresholdDecider : Prevents allocation to nodes that exceed disk watermarks ( cluster.routing.allocation.disk.watermark.low, high, threshold_enabled).

EnableAllocationDecider : Enables or disables allocation/rebalancing at cluster or index level via settings like cluster.routing.allocation.enable and index.routing.allocation.enable.

FilterAllocationDecider : Applies required, include, and exclude filters to control which nodes may receive shards.

MaxRetryAllocationDecider : Stops allocating a shard after a configurable number of failed retries ( index.allocation.max_retry).

NodeVersionAllocationDecider : Prevents allocation to nodes with incompatible versions.

RebalanceOnlyWhenActiveAllocationDecider : Allows rebalancing only when all replica shards are active.

ReplicaAfterPrimaryActiveAllocationDecider : Allows replica allocation only after the primary shard is active.

ResizeAllocationDecider : Ensures shards for resize operations are allocated next to source primaries.

RestoreInProgressAllocationDecider : Blocks allocation of shards that are currently being restored from snapshots.

SameShardAllocationDecider : Prevents multiple copies of the same shard from being allocated to the same host.

ShardsLimitAllocationDecider : Limits the total number of shards per node or per index.

SnapshotInProgressAllocationDecider : Prevents movement of shards that are part of an ongoing snapshot.

ThrottlingAllocationDecider : Controls the number of concurrent shard recoveries per node via cluster.routing.allocation.node_initial_primaries_recoveries and cluster.routing.allocation.node_concurrent_recoveries.

The AllocationService orchestrates node allocation, holding an AllocationDeciders composite and invoking the reroute method, which ultimately calls ShardsAllocator.allocate. The default allocator is BalancedShardsAllocator , which uses a weight function based on shard and index balance parameters ( cluster.routing.allocation.balance.shard, cluster.routing.allocation.balance.index, cluster.routing.allocation.balance.threshold) to compute node weights and achieve a balanced distribution.

The core weight calculations are:

weightindex(node, index) = indexBalance * (node.numShards(index) - avgShardsPerNode(index))
weightnode(node, index) = shardBalance * (node.numShards() - avgShardsPerNode)
weight(node, index) = weightindex(node, index) + weightnode(node, index)

BalancedShardsAllocator#allocate performs three main tasks: allocating unassigned shards, relocating shards that cannot stay on their current nodes, and rebalancing shards to improve overall cluster balance.

Reroute is triggered by index CRUD operations, configuration changes, or cluster state transitions.