MyFlash: An Open‑Source MySQL Binlog Flashback Tool and Its Architecture

Accidental data deletions caused by DBA mistakes or application bugs are common in production environments. Traditional recovery methods—manual reconstruction from logs, binlog replay, or backup restoration—are time‑consuming and error‑prone. The MySQL community introduced a flashback extension to mysqlbinlog, and Meituan‑Dianping further developed a dedicated tool called MyFlash to address these pain points.

The tool is now open‑source at https://github.com/Meituan-Dianping/MyFlash .

Existing recovery utilities can be grouped into three categories:

Using mysqlbinlog together with sed / awk – simple but fragile for complex data types.

Applying patches to MySQL source to add a flashback option – high performance but requires deep knowledge of MySQL replication internals and is version‑sensitive.

Leveraging third‑party binlog parsing libraries (e.g., binlog2sql) – easy to use but often slower and limited in filtering capabilities.

An ideal flashback tool should:

Parse binlog directly without converting it to text.

Support native filtering by database, table, SQL type, position, time, etc.

Be compatible with multiple MySQL versions.

Remain robust against internal code refactoring.

Provide full control over binlog parsing for maximum flexibility.

MyFlash starts with a concise overview of the binlog format. A complete binlog consists of a Format Description Event header, a series of data events, and a final Rotate Event. Common events include:

170905 01:59:33 server id 10 end_log_pos 123 CRC32 0xed1ec563
Start: binlog v 4, server v 5.7.18-log created 170905 01:59:33

170905 01:59:33 server id 10 end_log_pos 339 CRC32 0x3de40c0d
Table_map: `test`.`test4` mapped to number 238

170905 01:59:33 server id 10 end_log_pos 385 CRC32 0x179ef6dd
Update_rows: table id 238 flags: STMT_END_F
UPDATE `test`.`test4` WHERE @1=3 SET @1=13;

Rollback is achieved by swapping the event type codes (e.g., INSERT ↔ DELETE) and reversing UPDATE rows by exchanging before‑image (BI) and after‑image (AI) data. Accurate length calculation for each field (int, bigint, decimal, varchar, etc.) is essential; for example, a decimal(18,10) occupies 9 bytes (4 for the integer part and 5 for the fractional part).

Key parsing concepts include length‑encoded integers (variable‑length encoding to save space) and the storage format of decimal values.

MyFlash’s architecture consists of three stages:

Parsing binlog: The file is split into events and filtered by user‑specified time or position ranges.

Reassembling minimal execution units: Each unit groups a Table_map event with its associated row events, preserving execution order.

Generating the flashback binlog: The units are reversed, row events are inverted, and the resulting events are written back, updating next_position fields accordingly.

Performance tests on a table with 1,048,576 rows show that MyFlash outperforms both binlog2sql and the traditional mysqlbinlog ‑based approach, achieving the fastest recovery speed.

The article concludes with author bios (Guo You, Jin Long, Xing Fan – senior MySQL DBAs at Meituan‑Dianping) and a list of reference documents, including MySQL official internals documentation and related open‑source projects.