MTDDL: Meituan Distributed Data Layer Middleware Overview

Background : In Q3 2016, Meituan Waimai’s product‑2.0 launch caused a rapid increase in merchants and items, stressing the product database’s capacity and write‑peak QPS. The resulting issues included degraded query performance, DB master‑slave lag, and difficulty in schema changes.

Two solutions were considered: vertical splitting of the existing product table (short‑term) and sharding (database‑and‑table partitioning). Sharding was deemed inevitable.

Industry Research & Design Goal : After studying sharding solutions in other Meituan services and various middleware, the team decided to develop an in‑house client‑side sharding middleware named MTDDL (Meituan Distributed Data Layer). The design emphasizes generality, extensibility, comprehensive functionality, and easy integration.

Key Features :

Dynamic data source routing

Read‑write separation

Distributed unique ID generation

Database and table sharding

Connection‑pool management and SQL monitoring

Dynamic configuration

Logical Architecture : The article shows a DAO‑layer insert call sequence diagram illustrating ID generation, data‑source routing, and SQL tracing.

Implementation Details :

Dynamic Data Source & Read‑Write Separation : Extends Spring’s AbstractRoutingDataSource with a MultipleDataSource class. The routing is driven by a custom @DataSource annotation and AOP aspect ShardMultipleDataSourceAspect. Example configuration and DAO usage:

<bean id="multipleDataSource" class="com.sankuai.meituan.waimai.datasource.multi.MultipleDataSource">
    <property name="targetDataSources">
        <map key-type="java.lang.String">
            <entry key="dbProductWrite" value-ref="dbProductWrite"/>
            <entry key="dbProductRead" value-ref="dbProductRead"/>
        </map>
    </property>
</bean>

public interface WmProductSkuDao {
    @DataSource("dbProductWrite")
    void insert(WmProductSku sku);

    @DataSource("dbProductRead")
    WmProductSku getById(long sku_id);
}

Distributed Unique ID Generator : Integrates Meituan’s Leaf ID service, which allocates ID blocks from a DB‑backed ticket table. The middleware also allows custom ID strategies via the IDGenStrategy interface.

Sharding Algorithm : Default modulo strategy splits data across group_shard_num databases and table_shard_num tables. Core implementation:

public class ModStrategyHandle implements ShardStrategy {
    @Override
    public String getShardType() { return "mod"; }

    @Override
    public DataTableName handle(String tableName, String dataSourceKey, int tableShardNum,
                               int dbShardNum, Object shardValue) {
        long shard_value = Long.valueOf(shardValue.toString());
        long tablePosition = shard_value % tableShardNum;
        long dbPosition = tablePosition / (tableShardNum / dbShardNum);
        String finalTableName = tableName + "_" + tablePosition;
        String finalDataSourceKey = dataSourceKey + dbPosition;
        return new DataTableName(finalTableName, finalDataSourceKey);
    }
}

Custom sharding strategies can be added by implementing ShardStrategy.

Annotation‑Based Integration : Business code uses three annotations— @ShardInfo (sharding config), @ShardOn (shard key), and @IDGen (ID field). Example:

@DataSource("dbProductSku")
@ShardInfo(tableName="wm_food", tableShardNum=100, dbShardNum=1,
           shardType="mod", idGenType=IDGenType.LEAF, idGenKey=LeafKey.SKU)
@Component
public interface WmProductSkuShardDao {
    void insert(@ShardOn("wm_poi_id") @IDGen("id") WmProductSku sku);
    List<WmProductSku> getSkusByWmPoiId(@ShardOn long wm_poi_id);
}

Connection‑Pool & SQL Monitoring : Supports c3p0, dbcp1/2, etc., with metrics reported via Meituan’s JMonitor. SQL execution is traced using the MTrace component, providing latency, QPS, error rates, etc.

Dynamic Configuration : From version 1.0.3, MTDDL reads sharding and data‑source settings from Meituan’s MCC configuration center, allowing live updates without service restarts.

Version Evolution : Four development phases have been released, with plans to open‑source the project.