How to Migrate Elasticsearch Clusters to the Cloud: Snapshots, Logstash & elasticdump

1 COS Snapshot

When downtime is acceptable, use Elasticsearch snapshot API to create a repository in the source cluster, back up indices, and restore them in the target cluster. The target cluster’s major version must be equal to or higher than the source version.

Supported repository types include fs, url, s3, hdfs, and cos. For Tencent Cloud, install the cos-repository plugin and create a COS repository with a request such as:

PUT _snapshot/my_cos_backup
{
    "type": "cos",
    "settings": {
        "app_id": "xxxxxxx",
        "access_key_id": "xxxxxx",
        "access_key_secret": "xxxxxxx",
        "bucket": "xxxxxx",
        "region": "ap-guangzhou",
        "compress": true,
        "chunk_size": "500mb",
        "base_path": "/"
    }
}

Create a snapshot of all indices: PUT _snapshot/my_cos_backup/snapshot_1 Or snapshot specific indices:

PUT _snapshot/my_cos_backup/snapshot_2
{
    "indices": "index_1,index_2"
}

Check snapshot status with: GET _snapshot/my_cos_backup/snapshot_1 Restore the snapshot in the target cluster:

POST _snapshot/my_cos_backup/snapshot_1/_restore

To rename indices during restore, use:

POST _snapshot/my_cos_backup/snapshot_1/_restore
{
    "indices": "index_1",
    "rename_pattern": "index_(.+)",
    "rename_replacement": "restored_index_$1"
}

Monitor recovery with _recovery API and verify health with GET _cluster/health/index_1.

2 Logstash

Deploy Logstash in the same VPC as the target Elasticsearch cluster, ensuring it can reach the source cluster. Use a high‑performance VM (e.g., 16 CPU, 32 GB RAM) and match Logstash’s Elasticsearch version to the target cluster’s version.

Pay attention to index type differences across ES versions. A typical Logstash configuration for cross‑cluster migration is:

input {
    elasticsearch {
        hosts => "1.1.1.1:9200"
        index => "*"
        docinfo => true
        size => 5000
        scroll => "5m"
    }
}

output {
    elasticsearch {
        hosts => ["http://2.2.2.2:9200"]
        user => "elastic"
        password => "your_password"
        index => "%{[@metadata][_index]}"
        document_type => "%{[@metadata][_type]}"
        document_id => "%{[@metadata][_id]}"
    }
}

This configuration syncs all indices from the source to the target; you can restrict it to specific indices as needed.

3 elasticsearch‑dump

elasticsearch‑dump is an open‑source Node.js tool for ES data migration. Install it globally: npm install elasticdump -g Key parameters: --input: source URL, file, or stdin (e.g., http://host:9200/index) --output: destination URL, file, or stdout --type: data, settings, mapping, analyzer, alias (default: data)

Example: migrate a single index’s settings, mapping, and data:

elasticdump --input=http://172.16.0.39:9200/companydatabase --output=http://172.16.0.20:9200/companydatabase --type=settings
elasticdump --input=http://172.16.0.39:9200/companydatabase --output=http://172.16.0.20:9200/companydatabase --type=mapping
elasticdump --input=http://172.16.0.39:9200/companydatabase --output=http://172.16.0.20:9200/companydatabase --type=data

To migrate all indices (data only):

elasticdump --input=http://172.16.0.39:9200 --output=http://172.16.0.20:9200

Note that elasticdump does not transfer index settings such as shard count; those must be recreated manually.

Comparison of Methods

COS Snapshot : Ideal for large‑scale migrations (GB‑TB‑PB) with high speed requirements; works offline, suitable for cross‑cloud moves.

Logstash : Suitable for full or incremental migrations when both clusters are network‑reachable; offers flexible filtering but requires a VM in the same VPC.

elasticsearch‑dump : Best for small datasets; similar to mysqldump, logical backup and restore.

Summary

Logstash and elasticdump need network connectivity between source and target clusters; snapshot does not.

elasticsearch‑dump is comparable to mysqldump and fits small‑data scenarios.

Snapshot is the preferred method for large‑volume migrations.