How Facebook Overcame the Challenges of Migrating MySQL 5.6 to 8.0

Overview

Facebook announced that its recent major upgrade to MySQL 5.6 took more than a year, and that the team continued developing the MyRocks LSM‑tree storage engine on 5.6. To avoid slowing MyRocks progress, they decided to stay on 5.6 until MyRocks was complete and then upgrade directly to MySQL 8.0.

Migration Challenges

The upgrade required extensive effort for each new MySQL major version, including:

Porting custom features to the new version;

Ensuring replication compatibility between major releases;

Minimizing changes required by existing application queries;

Fixing performance regressions that prevented server support for Facebook workloads.

Code Patch Classification

More than 1,700 patches from the customized 5.6 branch needed to be ported to 8.0. Patches were grouped into four categories:

Drop : features no longer used or already present in 8.0;

Build/Client : non‑server features such as modified MySQL tools or new client APIs;

Non‑MyRocks Server : server‑side features unrelated to MyRocks;

MyRocks Server : features supporting the MyRocks storage engine.

Tracking spreadsheets recorded patch status and rationale for removals. Similar patches were bundled for migration, and each migrated patch retained its original 5.6 commit message for traceability.

Migration Path

Facebook combined multiple mysqld instances into MySQL replica sets, each with a primary and read‑only replicas distributed across data centers. The migration plan moved replica sets from a 5.6 primary/replica configuration to an 8.0 primary/replica configuration through the following steps:

Create an 8.0 replica from a logical dump of each 5.6 replica set;

Enable read traffic on the 8.0 replica;

Promote the 8.0 replica to primary;

Disable read traffic on the 5.6 primary;

Retire all 5.6 instances.

Each replica set could be migrated independently, allowing rollback at any step.

Row‑Based Replication

As part of the upgrade, Facebook adopted row‑based replication (RBR) as the default for MySQL 8.0 because many 8.0 features require it and it simplifies MyRocks migration. Existing replica sets using statement‑based replication (SBR) were gradually converted by adding primary keys where needed.

Automation Verification

Extensive automated testing validated mysqld behavior on 8.0. Tests uncovered issues such as log‑parsing failures, mismatched utf8mb4 collations, changed error codes for replication failures, missing table.frm handling in the new data dictionary, and the need to support dynamic privileges. Performance‑schema settings were trimmed to reduce memory consumption.

Application Verification

Shadow testing replayed production traffic on 8.0 test instances to catch query failures. Issues discovered included reserved keyword conflicts (e.g., groups , rank ), REGEXP incompatibilities, repeatable‑read deadlocks on INSERT … ON DUPLICATE KEY statements, and incompatibilities in custom document‑store and JSON functions. Some ACL cache contention and binlog rotation hotspots were also identified.

Future Work

The migration effort spanned several years, with most InnoDB replica sets now running on 8.0. Remaining work includes handling large logical dumps for massive instances, tracking API deprecations, supporting dual‑version replica sets during transition, and further optimizing memory usage. Despite the challenges, the upgrade delivered new features such as document store, improved date‑time support, and the potential for instant DDL on MyRocks.