Sync RDS MySQL Data to Alibaba Cloud Elasticsearch with DTS

Overview

Alibaba Cloud Data Transmission Service (DTS) provides real‑time data flow between relational databases, NoSQL stores, and OLAP systems. It combines synchronization, migration, subscription, integration, and processing, running on a dual‑active architecture that has been stable for seven years.

Prerequisites

A source RDS MySQL instance (see Alibaba Cloud quick‑start guide).

A target Elasticsearch instance (see Alibaba Cloud Elasticsearch creation guide).

The Elasticsearch storage capacity must exceed the storage used by the MySQL source.

Create a DTS Synchronization Task

Step 1 – Create the DTS instance

Use the Alibaba Cloud console to create a DTS instance and then add a new synchronization task.

Step 2 – Select source tables

Choose the MySQL tables you want to replicate to Elasticsearch.

Step 3 – Monitor task progress

The first run performs a full data load; subsequent runs handle incremental changes.

Validate Synchronization Results

By default DTS synchronizes both schema and full data. After the pre‑check, DTS copies the full dataset to the target cluster as a baseline for incremental updates. Once the full load finishes, you can query the synchronized data in Elasticsearch.

Querying the Synchronized Data with Kibana

Open the Kibana public entry page.

Log in with your credentials.

Navigate to Management > Dev Tools (the console).

Run the desired Elasticsearch queries.

Global Query

GET /mall_category/_search

The response shows the total number of documents (e.g., 1,041) and sample hits, confirming that the full sync succeeded.

Conditional Query

GET /mall_category/_search
{
  "query": {
    "match": {
      "name": "有商品无法删2"
    }
  }
}

This returns all documents whose name field matches the given term.

Precise (Phrase) Query

GET /mall_category/_search
{
  "query": {
    "match_phrase": {
      "name": "有商品无法删2"
    }
  }
}

The result contains a single document, demonstrating exact‑match capability.

Incremental Synchronization Test

Insert a new row into the MySQL mall_category table:

INSERT INTO `mall_category`
(`name`,`parent_id`,`is_show`,`image_url`,`sort_order`,`create_time`,`update_time`,`delete_time`)
VALUES ('开源技术小栈',1361,1,'https://img.tinywan.com/shop/img/2024-12/3a5cbd823.png',0,0,0,0);

After the insert, query Elasticsearch with a phrase query:

GET /mall_category/_search
{
  "query": {
    "match_phrase": {
      "name": "开源技术小栈"
    }
  }
}

The newly added document appears in the search results, confirming that DTS correctly propagated the incremental change.