Inside DBProxy: Open‑Source High‑Availability Database Middleware and Its Key Enhancements

Introduction

As data volumes grow, traditional single‑instance database access becomes a bottleneck. Database middleware enables horizontal scaling by routing requests to multiple MySQL instances, reducing load on any single server and decoupling DBA operations from development.

DBProxy Advantages

Supports multiple language MySQL clients

Read/write separation

Load balancing

Slave failure detection and removal (Master requires MHA or similar coordination)

Backend connection pooling

Custom SQL interception and filtering

Traffic grouping and control

Rich monitoring status

Sharding support (sharding version under internal testing)

Client IP restriction

Improvements Over Atlas

New parameters such as backend-max-thread-running and thread-running-sleep-delay to control per‑backend thread limits and wait times

Blacklist management based on SQL execution time and frequency, with dynamic add/remove commands and file persistence

Manual and automatic blacklist entry control via manual-filter-flag and auto-filter-flag Configurable thresholds: query-filter-time-threshold, query-filter-frequent-threshold, and access-num-per-time-window From‑replica traffic routing configuration

Dynamic variable handling and support for save config to add or delete sharding rules at runtime

Response‑time percentile statistics

Admin‑interface kill session support

Smooth backend online/offline transitions

Non‑root startup capability

Admin account security enhancements (password rotation, host restrictions)

Asynchronous log flushing with dedicated log thread

Graceful restart support

Thread‑running limits per MySQL backend with configurable timeout

Support for set transaction isolation level, use db, set option, and session‑level system variables

Connection‑property specification on client connect

Improved connection pool with idle‑connection timeout ( db-connection-idle-timeout)

Keep‑alive mechanism for client connections

Enhanced management and SQL logs, including implicit SQL statements and log rotation (controlled by sql-log-file-size and sql-log-file-num)

MySQL version reporting via mysql-version Performance boost: concurrent lexical analysis and elimination of redundant COM_INIT_DB when the database does not change, raising sysbench QPS from 70k to 220k in tests

Additional monitoring metrics (connection status, QPS, response time, network stats)

Dynamic SQL‑log configuration

Improved handling of autocommit‑false scenarios

Bug Fixes

Added keepalive and non‑blocking flags to sockets when establishing MySQL connections

RPM installation now creates configuration directory and default files automatically

RPM now adjusts proxy-dir without manual edits

Fixed client‑side disconnection when backend connections drop

Blocked accidental KILL statements on backend MySQL

Preserved backend connections during transaction errors to avoid incorrect rollbacks

Resolved character‑set errors in sharding result merging

Handled NULL values in sharding merge to prevent timeouts

Limited sharding column types in IN clause to int32

Fixed memory leak in connection cleanup (24‑byte leak)

Removed unnecessary admin logs and noisy messages

Corrected return values for set backend offline/online Addressed authentication errors and misconfigured DBProxy credentials

Fixed routing of SQL_CALC_FOUND_ROWS queries to master

Improved SQL parsing when comments are present

Prevented crashes caused by empty client strings

Vision and Community

The project aims to build a high‑availability, high‑reliability enterprise database middleware together with the community, inviting contributions via GitHub issues and a QQ user group.

Documentation and Resources

GitHub repository: https://github.com/Meituan-Dianping/DBProxy

Release notes: https://github.com/Meituan-Dianping/DBProxy/blob/master/doc/RELEASE_NOTES.md

Quick start: https://github.com/Meituan-Dianping/DBProxy/blob/master/doc/QUICK_START.md

User guide: https://github.com/Meituan-Dianping/DBProxy/blob/master/doc/USER_GUIDE.md

Programming guide: https://github.com/Meituan-Dianping/DBProxy/blob/master/doc/PROGRAMMING_GUIDE.md

Architecture & practices: https://github.com/Meituan-Dianping/DBProxy/blob/master/doc/THEORY_PRACTICES.md