JD Daojia MySQL Containerization: Architecture, Implementation, and Operational Practices

Background

With rapid growth of JD Daojia's business, MySQL traffic surged, making traditional cloud‑hosted MySQL instances unable to meet performance, reliability, and cost requirements. Physical machines were chosen and MySQL was deployed in Docker containers to address these challenges.

Technical Solution

The solution consists of three parts: a database resource platform, a monitoring system, and an automated operations platform.

Database Resource Platform

Hardware: 64‑core CPU, 256 GB RAM, 16 × 960 GB SSDs (RAID10) or 4 TB NVMe RAID0.

OS: CentOS 7.5.

Container runtime: Docker 1.13.1 in host‑network mode.

Images: custom MySQL 5.6.36 and MySQL 5.7.22.

Running MySQL in containers on physical hosts achieved up to 90 K QPS, far exceeding the 23 K QPS of cloud‑hosted instances.

Container Scheduling Algorithm

Distribute instances across different availability zones and hosts.

Limit the number of core‑business instances per host.

Separate shards of a sharding system onto different hosts.

Prefer hosts with the most free CPU, memory, and disk resources.

Allow CPU over‑commit up to 2× the physical cores.

A custom scheduler implements these rules to allocate containers efficiently.

MySQL High‑Availability

Domain‑based access is used; MHA and Zabbix are extended to enable rapid failover within 10 seconds. DNS records are updated and, to avoid cache delays, SaltStack pushes /etc/hosts changes to application servers for near‑instant resolution.

Monitoring System

Zabbix monitors each container’s CPU, memory, and other metrics via the Docker API, providing accurate data and triggering custom scripts for self‑healing actions such as disk‑space cleanup, high‑CPU alerts, and automatic DNS failover.

Automated Operations Platform

Built with Python and Flask, the platform automates the full MySQL lifecycle: resource request, container creation, master‑slave deployment, backup strategy selection, monitoring integration, and tool provisioning (Percona Toolkit, backup scripts, slow‑log processors, etc.). Container updates (CPU/memory) are performed via Docker update without downtime, enabling rapid scaling.

Summary

Over 95 % of JD Daojia’s MySQL instances now run in containers, delivering high performance, up to 50 % cost reduction compared to cloud VMs (and 100 % compared to cloud‑managed MySQL), and a five‑minute automated provisioning workflow that dramatically improves operational efficiency.