Troubleshooting xtrabackup Full Backup Failures Caused by Undo Log Truncation in MySQL 8.0
The article analyzes why xtrabackup full backups fail on MySQL 8.0 when undo log truncation occurs, identifies a bug triggered by the super_read_only setting, and provides reproducible steps and configuration tweaks to prevent the issue.
A customer reported three MySQL Group Replication (MGR) environments failing full backups with MySQL 8.0.18 and xtrabackup 8.0.9, showing errors such as "Cannot recover a truncated undo tablespace in read‑only mode".
xtrabackup: Generating a list of tablespaces
Directories to scan '.;./;.'
Scanning './'
Completed space ID check of 2 files.
Allocated tablespace ID 12 for zxc/a, old maximum was 0
Undo tablespace number 1 was being truncated when mysqld quit.
Cannot recover a truncated undo tablespace in read-only mode
xtrabackup: error: xb_load_tablespaces() failed with error code 57Searching Percona’s issue tracker linked to PXB‑1787 . The root cause is that undo‑log cleanup changes the undo log’s space_id , causing xtrabackup to abort.
The immediate fix is to restart the replica so the undo log is restored, then retry the backup. However, repeatedly restarting is not a sustainable solution.
Log inspection shows undo‑log truncation started at 00:48, while the backup began at 01:00. The innodb_max_undo_log_size was set to 1 GB, yet the undo logs ( undo_1_trunc.log and undo_2_trunc.log ) persisted for over ten hours.
2021-10-26T00:48:41.543308+08:00 0 [Note] [MY-012994] [InnoDB] Truncating UNDO tablespace 'innodb_undo_001'.
2021-10-26T00:57:25.369852+08:00 0 [Note] [MY-012994] [InnoDB] Truncating UNDO tablespace 'innodb_undo_002'.
2021-10-26T01:02:01.994594+08:00 11751 [Warning] [MY-012111] [InnoDB] Trying to access missing tablespace 4294967152
2021-10-26T01:02:01.994645+08:00 11751 [Warning] [MY-012111] [InnoDB] Trying to access missing tablespace 4294967151Reproducing the issue in a test environment by setting innodb_max_undo_log_size to 10 MB on both primary and replica, then generating load with sysbench, shows the replica creates undo_1_trunc.log and undo_2_trunc.log that remain after the load stops, while the primary cleans undo logs quickly.
# Set undo log size limit
set global innodb_max_undo_log_size=10485760;
# Run sysbench load
sysbench oltp_read_write --mysql-host=10.186.63.54 --mysql-port=3306 --mysql-user=root --mysql-password=123456 --mysql-db=sysbench --tables=6 --table-size=1000000 --report-interval=1 --threads=1 --time=300 runRunning a full backup with xtrabackup again reproduces the same error. Checking SHOW ENGINE INNODB STATUS\G shows no long‑running transactions, indicating the problem is not due to active undo usage.
Further investigation reveals a MySQL bug (Bug 104573) where the super_read_only parameter triggers undo‑log truncation failures. Temporarily setting super_read_only=0 on the replica resolves the backup issue.
To avoid the bug, increase innodb_max_undo_log_size so that undo logs are purged before reaching the truncation threshold, preventing the undo‑log truncate path that leads to the xtrabackup failure.
Key takeaways:
Undo‑log truncation can break xtrabackup full backups when super_read_only is enabled.
Restarting the replica or disabling super_read_only are immediate work‑arounds.
Setting a larger innodb_max_undo_log_size reduces the chance of hitting the bug.
Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.