Master Elasticsearch Performance: Memory, CPU, Shards, and Cluster Tuning

Memory

Elasticsearch runs on the JVM, so heap size must be set carefully. Allocate up to 50 % of the host RAM but never exceed 32 GB (recommended maximum 31 GB) to leave space for the OS file cache used by Lucene. Over‑allocating heap leads to long garbage‑collection pauses.

Typical configuration methods:

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms16g
-Xmx16g

or via launch parameters:

ES_JAVA_OPTS="-Xms10g -Xmx10g" ./bin/elasticsearch

CPU

Complex queries and heavy indexing demand ample CPU. Elasticsearch creates several thread pools per node; each pool has a queue that buffers incoming requests. Dynamic allocation is usually sufficient, so manual changes to pool sizes are not recommended unless a specific need arises. See the official thread‑pool documentation for details.

Shard Count

Shards are the unit of data distribution. Too many small shards increase coordination overhead, while too few large shards can overload the master node. Recommended shard size is 30 GB–50 GB. Primary shard count is fixed at index creation; changing it requires reindexing.

Guidelines:

Balance the number of shards against expected document volume.

Allocate sufficient resources to master nodes when shard count is high.

Replicas

Replicas provide high availability and can improve query throughput by serving reads. The default replica count is 1; increase only if the use case demands higher fault tolerance, keeping in mind the additional storage cost.

Hot‑Warm Architecture

Separate hot nodes (SSD, high‑CPU) for frequently accessed data from warm/cold nodes (HDD) for infrequently accessed, read‑only indices. Use tools like Curator or ILM to move indices between node types on a schedule. Typical recommendation: at least three hot nodes and three warm nodes for high availability.

Node Role Configuration

Define node roles in elasticsearch.yml:

node.master: true
node.data: false

Data node:

node.master: false
node.data: true

Coordinating node:

node.master: false
node.data: false

Troubleshooting Tips

Monitor heap usage, CPU load, and disk I/O. High heap usage (>75 %) triggers longer GC pauses; near‑100 % can cause aggressive GC and severe latency. Track non‑heap memory growth to avoid kernel OOM. Set alerts for CPU spikes, heap pressure, and disk I/O saturation.

Cache and Refresh Settings

Filters are cached as bitsets, speeding up repeated queries. Use filter clauses to benefit from this cache. Adjust refresh_interval based on write intensity; frequent writes may require a longer interval.

Additional Safeguards

Enable slow‑query logging to identify expensive queries.

Increase ulimit (e.g., ulimit -n 65535) to allow more open files.

Set bootstrap.mlockall: true to lock memory.

Disable destructive wildcard deletes with action.destructive_requires_name: true.

Monitoring APIs

Cluster health: GET _cluster/health?pretty Indices list: GET _cat/indices?pretty&v Node stats: GET _nodes?pretty Master node: GET _cat/master?pretty&v Cluster stats: GET _stats?pretty Node stats: GET _nodes/stats?pretty Use tools like Kibana or Cerebro to visualize these metrics continuously.

Conclusion

Elasticsearch ships with sensible defaults for newcomers, but production workloads require careful tuning of memory, CPU, shard layout, replicas, node roles, and monitoring to meet performance and reliability goals.