ShardingSphere Overview and Spring Boot Integration for Database Sharding, Proxy, and Sidecar

1. Introduction to ShardingSphere

ShardingSphere is an open‑source distributed database middleware ecosystem consisting of three independent products: Sharding‑JDBC, Sharding‑Proxy and the planned Sharding‑Sidecar. It provides standardized data sharding, distributed transactions and governance, suitable for Java homogeneous or heterogeneous environments, containers and cloud‑native scenarios.

It is positioned as a relational‑database middleware that leverages existing RDBMS capabilities rather than replacing them, co‑existing with NoSQL and NewSQL solutions.

1. Sharding‑JDBC

Lightweight Java framework that works at the JDBC layer, requiring only a JAR on the client side. It is compatible with any JDBC‑compliant database and ORM framework such as JPA, Hibernate, MyBatis, Spring JDBC Template.

Works with any third‑party connection pool (DBCP, C3P0, Druid, HikariCP, etc.).

Supports MySQL, Oracle, SQL Server and PostgreSQL.

2. Sharding‑Proxy

Transparent database proxy that encapsulates the binary protocol, currently offering a MySQL‑compatible version. It can be accessed by any MySQL client and is fully transparent to applications.

Acts like a native MySQL server.

Works with any MySQL‑compatible client.

3. Sharding‑Sidecar (planned)

Intended as a cloud‑native database proxy for Kubernetes or Mesos, deployed as a DaemonSet to intercept all database traffic, forming a “Database Mesh”. It aims to mesh distributed data‑access applications with databases in a zero‑intrusion manner.

2. Why Use ShardingSphere

Relational databases suffer performance degradation when index depth grows and when high concurrency creates a bottleneck. Sharding distributes load across multiple nodes, improving availability and reducing operational costs associated with large single‑instance databases.

Service‑oriented stateless architectures can scale cheaply, but the database often becomes the limiting factor; sharding alleviates this pressure.

When a single instance exceeds about 1 TB, backup and recovery become costly; sharding keeps each node within a manageable size.

Although NoSQL offers native distribution, its lack of SQL compatibility and ecosystem maturity make ShardingSphere a pragmatic bridge that retains the strengths of relational databases.

3. Data Sharding

Horizontal (range) sharding splits a table’s rows across multiple databases or tables based on a sharding key, e.g., using the remainder of a primary key or user ID to determine the target shard. The article illustrates this with a banking‑window analogy and an order‑id example.

4. Spring Boot Integration

1. Create databases ds0 and ds1 and tables t_order0‑t_order2

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for t_order0
-- ----------------------------
DROP TABLE IF EXISTS `t_order0`;
CREATE TABLE `t_order0` (
  `order_id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `order_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
  PRIMARY KEY (`order_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Compact;

-- ----------------------------
-- Table structure for t_order1
-- ----------------------------
DROP TABLE IF EXISTS `t_order1`;
CREATE TABLE `t_order1` (
  `order_id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `order_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
  PRIMARY KEY (`order_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Compact;

-- ----------------------------
-- Table structure for t_order2
-- ----------------------------
DROP TABLE IF EXISTS `t_order2`;
CREATE TABLE `t_order2` (
  `order_id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `order_name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
  PRIMARY KEY (`order_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Compact;

SET FOREIGN_KEY_CHECKS = 1;

2. Add Maven dependencies

org.apache.shardingsphere
shardingsphere-jdbc-core-spring-boot-starter
5.0.0
com.baomidou
mybatis-plus-boot-starter
3.5.2
com.mysql
mysql-connector-j
org.projectlombok
lombok
log4j
log4j
1.2.17
cn.hutool
hutool-all
5.8.18

3. application.yml configuration

spring:
  shardingsphere:
    props:
      # Print SQL statements
      sql-show: true
    datasource:
      # ds0 configuration
      ds0:
        driver-class-name: com.mysql.cj.jdbc.Driver
        jdbc-url: jdbc:mysql://localhost:3306/ds0?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=GMT&allowPublicKeyRetrieval=true
        username: root
        password: 2020
        type: com.zaxxer.hikari.HikariDataSource
      # ds1 configuration
      ds1:
        driver-class-name: com.mysql.cj.jdbc.Driver
        jdbc-url: jdbc:mysql://localhost:3306/ds1?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=GMT&allowPublicKeyRetrieval=true
        username: root
        password: 2020
        type: com.zaxxer.hikari.HikariDataSource
      names: ds0,ds1
    # Sharding rules
    rules:
      sharding:
        sharding-algorithms:
          custom-db-inline:
            type: INLINE
            props:
              algorithm-expression: ds$->{user_id%2}
          custom-table-inline:
            type: INLINE
            props:
              algorithm-expression: t_order$->{order_id%3}
        tables:
          t_order:
            actual-data-nodes: ds$->{0..1}.t_order$->{0..2}
            database-strategy:
              standard:
                sharding-algorithm-name: custom-db-inline
                sharding-column: user_id
            table-strategy:
              standard:
                sharding-algorithm-name: custom-table-inline
                sharding-column: order_id
async:
  executor:
    thread:
      core_pool_size: 5
      max_pool_size: 20
      queue_capacity: 90000
      name:
        prefix: async-
mybatis-plus:
  global-config:
    db-config:
      id-type: assign_id

4. Project structure (Order entity, mapper, service)

// Order entity
@Data
@TableName("t_order")
@SuppressWarnings("serial")
public class Order extends Model
{
    @TableId(type = IdType.ASSIGN_ID)
    private Long orderId;
    private Integer userId;
    private String orderName;
    @Override
    public Serializable pkVal() {
        return this.orderId;
    }
}

// Mapper
@Mapper
public interface OrderMapper extends BaseMapper
{}

// Service interface
public interface OrderService extends IService
{}

// Service implementation
@Service
public class OrderServiceImpl extends ServiceImpl
implements OrderService {}

5. Thread‑pool configuration (ExecutorConfig)

@Configuration
public class ExecutorConfig {
    @Value("${async.executor.thread.core_pool_size}")
    private int corePoolSize;
    @Value("${async.executor.thread.max_pool_size}")
    private int maxPoolSize;
    @Value("${async.executor.thread.queue_capacity}")
    private int queueCapacity;
    @Value("${async.executor.thread.name.prefix}")
    private String namePrefix;

    @Bean(name = "asyncServiceExecutor")
    public Executor asyncServiceExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(corePoolSize);
        executor.setMaxPoolSize(maxPoolSize);
        executor.setQueueCapacity(queueCapacity);
        executor.setThreadNamePrefix(namePrefix);
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        executor.initialize();
        return executor;
    }
}

6. Async service interface and implementation

public interface AsyncService {
    void add(List
orderList, CountDownLatch countDownLatch);
}

@Service
@Slf4j
public class AsyncServiceImpl implements AsyncService {
    @Resource
    private OrderServiceImpl orderService;

    @Async("asyncServiceExecutor")
    @Transactional(rollbackFor = Exception.class)
    @Override
    public void add(List
orderList, CountDownLatch countDownLatch) {
        try {
            log.debug(Thread.currentThread().getName() + "开始插入数据");
            orderService.saveBatch(orderList);
            log.debug(Thread.currentThread().getName() + "插入数据完成");
        } finally {
            countDownLatch.countDown();
        }
    }
}

7. Application entry point

@SpringBootApplication
@EnableAsync
@EnableTransactionManagement
public class ShardingSphereApplication {
    public static void main(String[] args) {
        SpringApplication.run(ShardingSphereApplication.class, args);
    }
}

8. Async controller

@RestController
public class AsyncController {
    @Autowired
    private AsyncService asyncService;

    @GetMapping("/test")
    public String testAsyncInsert() {
        CountDownLatch c = null;
        try {
            List
data = getData();
            List
> partition = ListUtil.partition(data, 2000);
            c = new CountDownLatch(partition.size());
            for (List
list : partition) {
                asyncService.add(list, c);
            }
            c.await();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            System.out.println("所有的数据插入完毕");
        }
        return "执行完毕";
    }

    private List
getData() {
        List
list = new ArrayList<>();
        for (int i = 0; i < 10000; i++) {
            Order o = new Order();
            o.setOrderName("苹果" + i);
            o.setUserId(i + 1);
            list.add(o);
        }
        return list;
    }
}

The controller generates 10,000 sample orders, partitions them into batches of 2,000, and inserts them asynchronously using the configured sharding rules. After all threads finish, a completion message is printed.