Rainbow Bridge: A ShardingSphere‑Based Database Middleware for Scalability and Reliability

As the DeWu App rapidly expands, its underlying databases face increasing pressure from diverse business lines that require data sharding, read‑write separation, and shadow‑DB routing. To address these needs, DeWu built a unified middleware called “Rainbow Bridge”, based on Apache ShardingSphere.

Technical selection : After evaluating open‑source middlewares such as Mycat, ShardingSphere, Kingshard, and Atlas, ShardingSphere was chosen for its plug‑in architecture, multi‑product stack (JDBC, Proxy, Sidecar), and strong community support.

Current capabilities (Proxy mode) :

Data sharding (horizontal, with built‑in algorithms like modulo, hash, time‑range, inline expressions; custom Groovy scripts supported).

Read‑write separation using SQL analysis to route reads to replicas and writes to the primary.

Shadow‑DB pressure testing by routing marked SQL to isolated shadow instances.

Rate limiting and circuit breaking per DB, table, SQL, or DML dimension to protect against overload.

Traffic correction in active‑active deployments via SQL hints based on user‑ID and data‑center rules.

Improvements made on top of ShardingSphere :

Usability : Integrated rule configuration into a graphical console, added validation, auditing, connection‑pool governance, and a client library that automatically injects Hint information.

Stability : Deployed multiple Proxy clusters with domain‑based isolation, introduced per‑logic‑library thread‑pool isolation, and added fast‑fail circuit‑breaker logic using Sentinel.

Observability : Added fine‑grained metrics (QPS, latency, errors at DB/table level), connection‑pool statistics, thread‑pool usage, and built dashboards. Implemented full‑link tracing via RAL statements or SQL comments.

SQL Insight : Planned collection of logical and physical SQL logs for real‑time query analysis.

Hybrid JDBC & Proxy architecture : Enabled seamless switching between Proxy and JDBC modes via the console and a custom Rainbow connection pool, providing a lossless fallback when all Proxy nodes fail.

Current challenges : Merging upstream ShardingSphere releases (5.0.0‑alpha to 5.1.2) without breaking existing business, and contributing internal enhancements back to the open‑source community.

The article concludes that while ShardingSphere’s source code is sophisticated, it requires careful reading and adaptation for specific scenarios, and the DeWu team looks forward to further collaboration with the ShardingSphere community.