Mastering Elasticsearch Data Sync and Cluster Architecture: Strategies & Best Practices

1. Elasticsearch Data Synchronization

1.1 Data sync problem

Elasticsearch stores hotel data from MySQL, so any change in MySQL must be reflected in Elasticsearch. In microservice architectures, the service handling MySQL and the service handling Elasticsearch may run in separate services, raising synchronization challenges.

1.2 Sync method 1: Synchronous call

1.3 Sync method 2: Asynchronous notification

1.4 Sync method 3: Binlog listener

1.5 Comparison of three sync methods

Method 1 – Synchronous call

Advantage: Simple and straightforward

Disadvantage: High service coupling

Method 2 – Asynchronous notification

Advantage: Low coupling, moderate implementation difficulty

Disadvantage: Relies on MQ reliability

Method 3 – Binlog listener

Advantage: Completely decouples services

Disadvantage: Increases DB load, higher implementation complexity

2. Elasticsearch Cluster

2.1 Cluster structure

Single‑node Elasticsearch faces massive data storage and single‑point‑failure issues. Data is split into N shards stored across multiple nodes, and each shard is replicated on other nodes.

2.2 Building a cluster

Shard and replica counts are defined when creating an index and cannot be changed later.

PUT /itcast
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      // mapping definition ...
    }
  }
}

2.3 Node roles

Elasticsearch nodes have distinct responsibilities:

master eligible : candidate master node, manages cluster state and shard allocation.

data : stores data, handles search, aggregation, CRUD.

ingest : preprocesses data before storage.

coordinating : routes requests, merges results, returns to client.

2.4 Distributed query

Each node role should be deployed separately for optimal performance.

2.5 Split‑brain issue

If the master node fails and a network partition occurs, multiple masters may be elected, causing a split‑brain. Elasticsearch avoids this by requiring a majority of eligible master votes; the setting discovery.zen.minimum_master_nodes (default in ES 7+) mitigates the problem.

2.6 Summary 1

Master‑eligible nodes participate in master election and manage cluster state; data nodes handle CRUD operations; coordinating nodes route requests and merge results.

2.7 Distributed storage

When adding a document, the coordinating node hashes the document ID (or routing value) and assigns it to a shard: shard = hash(_routing) % number_of_shards _routing defaults to the document ID; shard count is fixed at index creation.

2.8 Distributed query phases

Elasticsearch query consists of two phases:

Scatter phase: coordinating node distributes the request to each shard.

Gather phase: coordinating node collects and merges results from data nodes.

2.9 Summary 2

Document routing uses hash modulo shard count; query has scatter and gather phases.

2.10 Fault tolerance

The master monitors node health; if a node crashes, its shards are reallocated to other nodes, ensuring data safety.

After master failure, eligible masters elect a new master, and data migration proceeds.