Practical Elasticsearch Operations and Performance Tuning Guide

This article builds on six earlier posts and provides a concise, hands‑on guide for Elasticsearch operations and performance tuning, aiming to help practitioners solve real‑world issues.

1. View Unassigned Shards

GET _cat/shards?v&h=index,shard,prirep,state,unassigned.reason&s=state:asc

2. Dynamically Adjust Replica Count

PUT my-index-2021.05.30-000002/_settings
{
  "number_of_replicas": 0
}

Note: Primary shards cannot be changed (except via shrink), but replicas can be adjusted on the fly.

3. Re‑enable Shard Allocation

PUT /_cluster/settings
{
  "transient": {
    "cluster.routing.allocation.enable": "all"
  }
}

4. Manually Move Unassigned Shards

POST /_cluster/reroute
{
  "commands": [
    {
      "move": {
        "index": "test",
        "shard": 0,
        "from_node": "node1",
        "to_node": "node2"
      }
    },
    {
      "allocate_replica": {
        "index": "test",
        "shard": 1,
        "node": "node3"
      }
    }
  ]
}

5. Check Disk Usage

GET /_cat/allocation?v

If usage reaches 85% or higher, the cluster is at a warning threshold and should be alerted.

6. View Node Versions

GET /_cat/nodes?v&h=host,name,version

Version mismatches across nodes can cause unpredictable errors.

7. Search Performance Tuning

7.1 Slow Log Settings

PUT /my-index-000001/_settings
{
  "index.search.slowlog.threshold.query.warn": "10s",
  "index.search.slowlog.threshold.query.info": "5s",
  "index.search.slowlog.threshold.query.debug": "2s",
  "index.search.slowlog.threshold.query.trace": "500ms",
  "index.search.slowlog.threshold.fetch.warn": "1s",
  "index.search.slowlog.threshold.fetch.info": "800ms",
  "index.search.slowlog.threshold.fetch.debug": "500ms",
  "index.search.slowlog.threshold.fetch.trace": "200ms",
  "index.search.slowlog.level": "info"
}

7.2 Mapping with Required Routing

PUT my-index-000002
{
  "mappings": {
    "_routing": {
      "required": true
    }
  }
}

7.3 Force Merge Segments

POST /my-index-000001/_forcemerge

8. Write Optimization

8.1 Bulk Write

POST _bulk
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : { "_id" : "1", "_index" : "test" } }
{ "doc" : { "field2" : "value2" } }

8.2 Increase Refresh Interval

PUT /my-index-000001/_settings
{
  "index": {
    "refresh_interval": "30s"
  }
}

8.3 Set Replicas to Zero for Faster Writes

PUT my-index-000001/_settings
{
  "number_of_replicas": 0
}

8.4 Asynchronous Translog Flush

PUT my-index-2021.06.03/_settings
{
  "index": {
    "translog": {
      "durability": "async"
    }
  }
}

9. Heap Memory Tuning

Heap size must be set in the jvm.options file and requires a restart to take effect.

ES_HEAP_SIZE=DESIRED_SIZE (e.g., "3g")

10. Disk‑Space Shortage Solutions

When disk usage exceeds critical thresholds, Elasticsearch blocks write operations, affecting cluster performance. Solutions include:

Horizontal scaling: add data nodes with balanced shard allocation.

Vertical scaling: upgrade hardware or increase disk capacity, adjusting path.data if needed.

Data migration: move stale or archived data to another cluster or storage.

Conclusion

Performance tuning is an ongoing effort; refer to the official “How‑to” documentation for detailed guidance on write and search optimizations.

This summary focuses on command‑line troubleshooting; for deeper performance improvements, consult the linked practical articles.