From Middleware to Distributed Database: UDDB’s Evolution and Architecture

UDDB’s Technical Evolution Path

Combining UDDB’s functional features and product philosophy, the development direction is a distributed database on public cloud built on database middleware. The roadmap starts with a simple engineering implementation, then leverages rapid cloud iteration to enhance business support, ultimately evolving into a big‑data era distributed database solution for internet, IoT, and traditional industry transformation.

The evolution follows three stages: 1) achieve 100% MySQL compatibility for read‑write separation with a master‑slave architecture, requiring no changes to tables or code; 2) support vertical sharding by routing SQL to appropriate MySQL instances with minimal table relocation; 3) gradually add horizontal sharding support, aligning with mainstream middleware features and later enhancing SQL and transaction handling.

UDDB’s System Architecture

The architecture consists of three main modules:

UDB Resource Pool – shares resources with UDB products; UDDB storage nodes and read‑only instances reuse the same availability zone.

Distributed Database Middleware System – includes Routerd, Mgrd, highly‑available UDB instances for metadata, and a Zookeeper cluster for decision‑making and scheduling.

UDDB Management Console – a web UI for creating, managing, and releasing UDDB instances.

From a business‑access perspective, client requests arrive via a load balancer (ULB) and are forwarded to Routerd, which parses the SQL, determines its type (DML or DDL), and routes it to the appropriate UDB nodes or broadcasts DDL changes through Mgrd via Zookeeper.

From a system‑management view, UDDB’s creation mirrors the typical workflow of assembling a distributed database using middleware, MySQL instances, and load‑balancing components, encapsulated in a user‑friendly console.

SQL Parsing and Routing Module (Routerd)

Routerd’s core is an SQL interpreter that parses syntax and semantics, identifies target UDB nodes, transforms the original SQL into sub‑SQLs, pushes them downstream, and aggregates results. A robust parser improves generality, allowing support for diverse client tools.

Common shortcomings in existing middleware include reliance on external parsers (e.g., SQLite, Alibaba Druid) and insufficient semantic analysis, leading to complex AST handling and limited support for advanced SQL features such as combined GROUP BY, ORDER BY, DISTINCT, LIMIT, and aggregate functions.

To improve Routerd’s universality, two steps are proposed: build an independent, standards‑compliant parser using Lex&Yacc, and adopt a database‑system‑style semantic analysis that separates syntax parsing, execution‑plan generation, optimization, and execution.

The execution plan is a concise, flat representation that facilitates sub‑SQL generation and result filtering/aggregation. Over a year of iteration, UDDB has progressed from single‑table support to multi‑table cross‑node joins and distributed transactions (expected in early 2018).

Technical Innovations

1. Modify the syntax‑parsing module to extract database and table names into a linked list while building the abstract syntax tree (AST).

2. After parsing, scan the list of extracted table names, determine whether each table is a regular table or a horizontally sharded table, and route the SQL accordingly: regular tables follow read‑write separation routing, while sharded tables undergo additional semantic analysis, execution‑plan generation, and execution.

This approach achieves near‑100% MySQL compatibility for read‑write separation and allows both read‑write separation and horizontal sharding to coexist within a single product, with minimal performance overhead (approximately 150 lines of code).

Conclusion

The article presents UDDB’s technical evolution and architecture, aiming to solve single‑node MySQL limitations and provide smoother business operations. Leveraging UCloud’s database kernel expertise, UDDB offers a complete middleware stack with independent syntax and semantic parsers, execution‑plan generation, and execution modules, supporting all major MySQL client tools. By mid‑2018, UDDB plans to add native distributed transaction and join support, completing the distributed storage, transaction, and SQL execution layers and delivering a truly distributed database that retains MySQL’s native deployment and operational experience.