Data Space Architecture and Metadata Models
The article outlines a data‑space architecture that employs a wide‑table design with dynamic columns and dedicated metadata tables, a metadata execution engine for business‑logic mapping, upgraded SQL parsing via Druid, MySQL‑proxy protocol handling, and distributed flow control using Redis and Zookeeper to enable scalable, multi‑tenant, low‑code and cloud‑native data management.
We discuss the architecture of data space, focusing on metadata storage models and SQL parsing engines.
The data space uses a wide table design with dynamic columns for flexible data storage, supported by metadata tables for objects, fields, and data tables.
The metadata execution engine maps business logic to metadata storage models, enabling multi-tenant data management.
SQL parsing involves upgrading from antlr to Druid for performance, with a command execution engine handling metadata mapping and SQL rewriting.
Protocol layer adaptations include MySQL proxy for standard protocol support and handling information_schema, system variables, and set/commit statements.
Key technical upgrades include SQL rewriting for metadata mapping, protocol layer enhancements, and distributed flow control using Redis and Zookeeper.
The architecture supports low-code platforms and cloud storage, with scalability and governance features.
DaTaobao Tech
Official account of DaTaobao Technology
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.