Understanding MySQL 8.0 Clone Plugin: Architecture, Implementation, and Comparison with Xtrabackup

Since MySQL 8.0.17, the official Clone feature allows users to copy a remote or local database instance with a simple SQL command, quickly launching a new instance.

1. Limitations of Xtrabackup Backup

During Xtrabackup, the speed of copying Redo Logs may not keep up with the production Redo Log generation. Because Redo Logs are circular, old logs can be overwritten before Xtrabackup finishes copying them, jeopardizing data consistency.

Redo Log working principle (source: cnblogs).

2. Basic Principles of Clone

The Clone plugin follows the design described in WL#9209 and performs the operation in five stages:

INIT : The clone object is created and identified by a locator.

FILE COPY : After calling snapshot_copy , the state changes to "FILE COPY" and all database files are copied to the caller while page tracking starts at the LSN "CLONE START LSN".

PAGE COPY : After all files are sent, the state changes to "PAGE COPY". Redo archiving starts at "CLONE FILE END LSN" and page tracking stops. Modified pages between the two LSNs are read from the buffer pool, sorted by space ID and page ID, and sent to the target.

REDO COPY : After page copy, the state moves to "REDO COPY". Redo archiving stops at "CLONE LSN" (the LSN of the cloned database). The redo logs from "CLONE FILE END LSN" to "CLONE LSN" are sent before reaching the final state.

Done : The clone object remains in this state until snapshot_end() destroys it.

The most critical phases are FILE COPY (physical copy of all InnoDB tablespace files with page‑tracking) and PAGE COPY (unique to Clone, handling dirty‑page collection and redo archiving).

3. Code Structure and Call Logic

SQL/Server Layer

sql/sql_lex.h

sql/sql_yacc.yy

sql_admin.cc – adds Clone syntax support.

clone_handler.cc – client handling of CLONE SQL and server handling of COM_XXX commands.

Plugin Layer

clone_plugin.cc – plugin interface and initialization.

clone_local.cc – concrete Clone operations.

clone_os.cc – OS‑level functions such as sendfile , read , write .

clone_hton.cc – interface with the storage engine.

clone_client.cc / clone_server.cc – client and server sides of Clone.

clone_status.cc – overall task progress; managed by Clone_Task_Manager .

InnoDB Engine Layer

Clone: storage/innobase/clone clone0clone.cc – clone task and runtime operation. clone0snapshot.cc – snapshot management. clone0copy.cc – file‑copy methods. clone0apply.cc – apply methods. clone0desc.cc – serialized data descriptor.

Archiver: storage/innobase/arch arch0arch.cc arch0page.cc arch0log.cc

Local Clone call stack (excerpt):

Clone_Handle::process_chunk(Clone_Task*, unsigned int, unsigned int, Ha_clone_cbk*) (/mysql-8.0.33/storage/innobase/clone/clone0copy.cc:1440)
Clone_Handle::copy(unsigned int, Ha_clone_cbk*) (/mysql-8.0.33/storage/innobase/clone/clone0copy.cc:1379)
innodb_clone_copy(handlerton*, THD*, unsigned char const*, unsigned int, unsigned int, Ha_clone_cbk*) (/mysql-8.0.33/storage/innobase/clone/clone0api.cc:561)
hton_clone_copy(THD*, std::__1::vector<myclone::Locator, std::__1::allocator<myclone::Locator>>&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>>&, Ha_clone_cbk*) (/mysql-8.0.33/plugin/clone/src/clone_hton.cc:152)
myclone::Local::clone_exec() (/mysql-8.0.33/plugin/clone/src/clone_local.cc:172)
myclone::Local::clone() (/mysql-8.0.33/plugin/clone/src/clone_local.cc:73)
plugin_clone_local(THD*, char const*) (/mysql-8.0.33/plugin/clone/src/clone_plugin.cc:456)
Clone_handler::clone_local(THD*, char const*) (/mysql-8.0.33/sql/clone_handler.cc:135)
Sql_cmd_clone::execute(THD*) (/mysql-8.0.33/sql/sql_admin.cc:2017)
mysql_execute_command(THD*, bool) (/mysql-8.0.33/sql/sql_parse.cc:4714)
dispatch_sql_command(THD*, Parser_state*) (/mysql-8.0.33/sql/sql_parse.cc:5363)
dispatch_command(THD*, COM_DATA const*, enum_server_command) (/mysql-8.0.33/sql/sql_parse.cc:2050)
do_command(THD*) (/mysql-8.0.33/sql/sql_parse.cc:1439)
handle_connection(void*) (/mysql-8.0.33/sql/conn_handler/connection_handler_per_thread.cc:302)
pfs_spawn_thread(void*) (/mysql-8.0.33/storage/perfschema/pfs.cc:3042)
_pthread_start (@_pthread_start:40)

4. Page Archiving System

Page Archiving, absent in Xtrabackup, reduces the amount of Redo Log copied during Clone by tracking dirty pages. Two collection schemes exist: (1) collect on mtr commit, (2) collect during purge flushing. The Clone plugin chooses scheme 2 to avoid blocking transaction commits.

The entry point for dirty‑page collection is buf_flush_page :

buf0flu.cc
if (flush) {
    mutex_enter(&buf_pool->flush_state_mutex);
    ...
    arch_page_sys->track_page(bpage, buf_pool->track_page_lsn, frame_lsn, false);
}

The tracking data is stored in arch_page_sys . When the block is full, a new block is allocated, potentially blocking the flush thread.

void Arch_Page_Sys::track_page(buf_page_t *bpage, lsn_t track_lsn, lsn_t frame_lsn, bool force) {
    Arch_Block *cur_blk;
    uint count = 0;
    ...
    arch_oper_mutex_enter();
    while (true) {
        if (m_state != ARCH_STATE_ACTIVE) break;
        cur_blk = m_data.get_block(&m_write_pos, ARCH_DATA_BLOCK);
        if (cur_blk->get_state() == ARCH_BLOCK_ACTIVE) {
            if (cur_blk->add_page(bpage, &m_write_pos)) break;
            cur_blk->end_write();
            m_write_pos.set_next();
            if (m_write_pos.m_block_num % ARCH_PAGE_FILE_DATA_CAPACITY == 0) {
                Arch_Block *reset_block = m_data.get_block(&m_reset_pos, ARCH_RESET_BLOCK);
                reset_block->end_write();
                m_reset_pos.set_next();
            }
            os_event_set(page_archiver_thread_event);
            ++count;
            continue;
        } else if (cur_blk->get_state() == ARCH_BLOCK_INIT || cur_blk->get_state() == ARCH_BLOCK_FLUSHED) {
            ut_ad(m_write_pos.m_offset == ARCH_PAGE_BLK_HEADER_LENGTH);
            cur_blk->begin_write(m_write_pos);
            if (!cur_blk->add_page(bpage, &m_write_pos)) ut_d(ut_error);
            break;
        } else {
            bool success;
            ...
            success = wait_flush_archiver(cbk);
            count = success ? 0 : 2;
            continue;
        }
    }
    arch_oper_mutex_exit();
}

The archiving thread page_archiver_thread writes blocks to disk, using a double‑write buffer when persisting:

void page_archiver_thread() {
    bool page_wait = false;
    while (true) {
        auto page_abort = arch_page_sys->archive(&page_wait);
        if (page_abort) {
            ib::info(ER_IB_MSG_14) << "Exiting Page Archiver";
            break;
        }
        if (page_wait) {
            os_event_wait(page_archiver_thread_event);
            os_event_reset(page_archiver_thread_event);
        }
    }
}

When flushing a block, the plugin may write to the double‑write buffer before persisting the archived data:

dberr_t Arch_Block::flush(Arch_Group *file_group, Arch_Blk_Flush_Type type) {
    switch (m_type) {
        case ARCH_RESET_BLOCK:
            err = file_group->write_file_header(m_data, m_size);
            break;
        case ARCH_DATA_BLOCK: {
            bool is_partial_flush = (type == ARCH_FLUSH_PARTIAL);
            err = file_group->write_to_file(nullptr, m_data, m_size, is_partial_flush,
                                          true, get_empty_file_header_cbk);
            break;
        }
        default:
            ut_d(ut_error);
    }
    return err;
}

5. Summary

Clone is more convenient than using Xtrabackup to start a slave.

Clone copies far fewer Redo Log entries than Xtrabackup, reducing failure risk; the arch_log_sys component controls log writes to avoid overwriting unarchived logs.

Clone forces a checkpoint at start and limits Redo Log archiving, which may impact a heavily loaded primary server.

References

"MySQL – Engine Features – An Introduction to Clone Plugin" (http://mysql.taobao.org/monthly/2019/09/02/)

"MySQL: Plugin Callback Mechanism" (https://greatsql.cn/blog-74-1158.html)

"MySQL – Engine Features – clone_plugin" (http://mysql.taobao.org/monthly/2019/08/05/)

"Practical MySQL 8.0.17 Clone Plugin" (https://opensource.actionsky.com/20190726-mysql/)

"Comprehensive Analysis of MySQL Clone Plugin Implementation" (https://zhuanlan.zhihu.com/p/433606318)

"MySQL/InnoDB Data Clone Plugin Implementation Dissection" (https://sq.sf.163.com/blog/article/364933037836570624)

"MySQL 8 New Feature – Clone Plugin" (https://www.cnblogs.com/ivictor/p/13818440.html)

Additional resources: GitHub repository https://github.com/actiontech/sqle, documentation https://actiontech.github.io/sqle-docs/, official site https://opensource.actionsky.com/sqle/, and commercial support https://www.actionsky.com/sqle.