How Apache ShardingSphere Evolved into a Distributed Database Middleware

With the rapid growth of the open‑source industry, open‑source technologies have become a key driver of digital transformation, attracting significant investment and corporate interest. Over the past year, more than ten domestic open‑source companies secured new financing, while major enterprises accelerated their open‑source initiatives and the first Chinese open‑source foundation was established.

In this context, Zhang Liang, architect at JD Technology and VP & founder of the Apache ShardingSphere project, shares the journey of ShardingSphere—from its humble beginnings as Sharding‑JDBC to becoming a top‑level Apache project—and discusses the challenges of enterprise open‑source governance and community operation.

Development Phases of ShardingSphere

Initially named Sharding‑JDBC, the project was personally led by Zhang Liang and implemented a database sharding and partitioning framework based on the Java JDBC interface. The first version quickly gained developer approval by abstracting the complexities of horizontal scaling.

After the success of Sharding‑JDBC, JD Technology invited Zhang Liang to join full‑time, leading to the second phase where the project was renamed ShardingSphere, expanding its scope beyond Java and attracting broader community interest.

In November 2018, ShardingSphere entered the Apache Incubator, and 17 months later graduated as a top‑level Apache project, completing the transition to community ownership and establishing a plug‑in architecture for further evolution.

Three‑Layer Plug‑in Architecture

L1 Kernel Layer: Focuses on the database kernel, including query optimization, distributed transactions, core execution chain, scheduling engine, and distributed governance. It aims to provide incremental middleware services during the early stages of distributed databases and later support full distributed query optimization and transaction management. L2 Function Layer: Targets enterprise applications, offering data sharding, read/write splitting, high availability, strong consistency, elastic migration, data encryption, shadow databases, and other extensions. This layer is extensible by developers and is now mature. L3 Ecosystem Layer: Serves the open‑source ecosystem, covering SQL dialects, database protocols, and data gateways, facilitating integration with heterogeneous data sources.

Five Practical Tips for Building a Sustainable Open‑Source Community

Listen and Observe: Use GitHub and mailing lists to capture even the smallest feedback, turning it into actionable improvements.

Maintain English Communication: Ensure the community remains inclusive by using English as the common language.

Make Issues Searchable: Encourage detailed, well‑tagged issue reports to aid future discoverability and knowledge sharing.

Implement Strong Testing: Require thorough testing before code acceptance to prevent overburdening reviewers.

Avoid Reinventing the Wheel: Leverage existing open‑source solutions and contribute back to prevent duplicate effort.

Zhang emphasizes that a vibrant community, not just code, is the true value of open source. He advocates for larger enterprises to allocate dedicated full‑time open‑source roles to foster stable, long‑term project health.