Five Ways to Sync MySQL Data to Elasticsearch, Redis, MQ, etc.

In real‑world applications we often need to synchronize MySQL data to other data stores so that any insert, update, or delete performed on MySQL is reflected in systems such as Elasticsearch, Redis, or message queues like Kafka.

1. Business‑Layer Sync

Since MySQL operations are performed in the business layer, it is natural to add synchronization code there as well, commonly by writing logic in ORM hooks.

The drawback is that as the number of services grows, synchronization code becomes scattered, making coordinated updates difficult—for example, an incompatible migration of an Elasticsearch index could affect many services.

2. Middleware Sync

When the architecture evolves into micro‑services, services may no longer call MySQL directly but go through a middleware layer. The middleware can synchronize other data sources while performing MySQL operations.

This approach requires the middleware to be adapted, adding some complexity.

3. Scheduled Task Based on updated_at Field

By adding a special column such as updated_at to the MySQL table, a scheduled job can query rows that have changed since the last run, achieving incremental updates.

Open‑source tools like Logstash can be used for this purpose.

The main limitation is that delete operations cannot be synchronized with this method.

4. Parse Binlog for Synchronization

Tools such as Canal act as a pseudo‑slave to read MySQL binary logs (binlog) and capture data changes.

This is a mature industry solution, but it requires setting the MySQL binlog-format to ROW mode.

5. Parse Binlog – Mixed / Statement Format

MySQL binlog can be in three formats: ROW – records changes row by row. statement – records the executed SQL statements. mixed – a combination of the above, recording either statements or row changes.

If the MySQL binlog cannot be set to ROW, you can still parse it, but you will receive raw SQL statements or row‑level changes that must be interpreted (e.g., via regex or AST parsing) before applying synchronization logic.

This approach requires custom business‑specific SQL parsing and can be difficult for bulk updates, though it works well for simple primary‑key‑based modifications or deletions.

Conclusion

Below are some open‑source libraries for binlog parsing:

Canal

go-mysql

Zongji