Step‑by‑Step Logstash Migration from Elasticsearch 9.0 to Easysearch with Zero Downtime

Project Background and Goals

Migration from Elasticsearch 9.0.0 on Tencent Cloud to Easysearch, a compatible Elasticsearch fork. Objectives: migrate full index data, keep document IDs unchanged, ensure data integrity, and minimize impact.

Technical Design

Solution Selection

Compared snapshot restore, Reindex API, Logstash, and third‑party tools; Logstash was chosen for its strong flexibility, real‑time monitoring, scroll‑based breakpoint resume, and cross‑version compatibility.

Architecture Overview

Core Configuration

Input Plugin

input {
  elasticsearch {
    hosts => ["https://X.X.X.X:9200"]
    index => "product-index"
    user => "elastic"
    password => "changeme"
    ssl_enabled => true
    ssl_verification_mode => "none"
    docinfo => true
    size => 500
    scroll => "5m"
  }
}

Key parameters: hosts (source HTTPS address), index (product-index), user / password, ssl_enabled true, ssl_verification_mode "none" (skip cert verification for self‑signed certs), docinfo true (preserve metadata), size 500 (documents per batch), scroll "5m" (scroll context).

Filter Plugin

filter {
  ruby {
    code => '
      # Extract index name and document ID from metadata
      if event.get("[@metadata][input][elasticsearch][_index]")
        event.set("[@metadata][_index]", event.get("[@metadata][input][elasticsearch][_index]"))
        event.set("[@metadata][_id]", event.get("[@metadata][input][elasticsearch][_id]"))
      end
    '
  }
}

The Ruby script extracts the original index name and document ID from Logstash metadata and stores them in [@metadata][_index] and [@metadata][_id], making them available to the output stage and ensuring ID consistency.

Output Plugin

output {
  elasticsearch {
    hosts => ["http://127.0.0.1:9200"]
    index => "%{[@metadata][_index]}"
    document_id => "%{[@metadata][_id]}"
    manage_template => false
    template_overwrite => false
    ilm_enabled => false
  }
}

Key parameters: hosts (target Easysearch address), dynamic index and document_id reuse the source values, and template/ILM management disabled.

Implementation Steps

Preparation

Verify connectivity to the source Elasticsearch cluster.

Verify connectivity to the target Easysearch cluster.

Execute Migration

.\logstash.bat -f "D:\software\logstash-9.0.0-windows-x86_64\logstash-9.0.0\config\migration.conf"

Verification

Data integrity is validated by comparing document counts and confirming that document IDs match before and after migration. The verification confirms a lossless migration.

Key Technical Points

Document ID Preservation

Setting docinfo => true together with the Ruby script guarantees that target documents retain the exact IDs from the source, preventing duplicates.

Batch Processing Optimization

size: 500 – balances network throughput and memory usage.

scroll: 5m – provides sufficient time for large‑volume batches.

Parameters can be tuned according to the actual network environment.

SSL Handling

ssl_enabled => true

– enables TLS encryption for the source connection. ssl_verification_mode => "none" – skips certificate verification (production should use valid certificates).

Conclusion

The Logstash‑based approach achieved a smooth, zero‑downtime migration from Elasticsearch 9.0 to Easysearch, preserving index data, document IDs, and overall integrity. The solution was validated in a production environment.