Pirate Middleware: Service Orchestration for Meituan Service Experience Platform

In the mobile Internet era, Meituan's Service Experience Platform aims to solve various customer problems during product selection, purchase, and usage by being truly customer‑centric. The platform originally only stored complaint data and had to rely on many business data sources (order, refund, product, etc.) provided by different business lines.

Problems

1. Wide and diverse integration scenarios : Hundreds of business lines (food delivery, hotel, travel, ride‑hailing, etc.) each have multiple sub‑lines, resulting in tens of thousands of data‑access scenarios that need to be integrated.

2. High customization requirements : Each business side needs custom logic tightly coupled with its data, forcing the platform’s RD to write and maintain a large amount of adapter code.

Early solution

The platform introduced an adaptation layer that abstracted all business data and provided standardized internal interfaces. All custom logic was implemented in adapters written by the platform’s RD. This approach required heavy coding effort and deep business knowledge from the platform side.

public class TicketAdapterServiceImpl implements OrderAdapterService {
    @Resource(name = "tradeQueryClient")
    private TradeTicketQueryClient tradeTicketQueryClient;
    @Resource
    private ColumbusTicketService columbusTicketService;
    /**
     * Get ticket‑related order data by order ID
     */
    @Override
    public OrderInfoDTO handle(OrderRequestDTO orderRequestDTO) {
        List<ITradeTicketQueryService.TradeDetailField> tradeDetailFieldList = new ArrayList<>();
        tradeDetailFieldList.add(ITradeTicketQueryService.TradeDetailField.ORDER);
        tradeDetailFieldList.add(ITradeTicketQueryService.TradeDetailField.TICKET);
        tradeDetailFieldList.add(ITradeTicketQueryService.TradeDetailField.REFUND_REQUEST);
        try {
            RichOrderDetail richOrderDetail = tradeTicketQueryClient.getRichOrderDetailById(orderRequestDTO.getOrderId(), tradeDetailFieldList);
            if (richOrderDetail == null || richOrderDetail.getOrderDetail() == null) {
                return null;
            }
            OrderDetail orderDetail = richOrderDetail.getOrderDetail();
            RefundDetail refundDetail = richOrderDetail.getRefundDetail();
            OrderInfoDTO orderInfoDTO = new OrderInfoDTO();
            orderInfoDTO.put("dealId", orderDetail.getMtDealId());
            orderInfoDTO.put(DomesticTicketField.VOUCHER_CODE.getValue(), getVoucherCode(richOrderDetail));
            // ... more field mapping ...
            ColumbusTicketDTO columbusTicketDTO = columbusTicketService.getByDealId((int) orderDetail.getMtDealId());
            if (columbusTicketDTO != null) {
                orderInfoDTO.put(DomesticTicketField.REFUND_INFO.getValue(), columbusTicketDTO.getRefundInfo());
                // ... more field mapping ...
            }
            return orderInfoDTO;
        } catch (TException e) {
            Cat.logError("查询不到对应的订单详情", e);
            return null;
        }
    }
}

Improved approach

The adaptation layer was handed over to the business side, reducing the platform RD’s workload but increasing the business side’s effort. To combine the advantages of both, a new middleware called Pirate was designed.

Pirate Middleware – Overview

Pirate is a service‑orchestration middleware that enables generic calls to existing services and assembles them via a JSON‑based DSL. It decouples the caller from service protocols and allows parallel or sequential execution.

package com.dianping.demo;
public interface DemoService {
    OrderDTO getById(String orderId);
}

public class DemoInvoke {
    public void genericInvoke() {
        /** configure generic caller **/
        InvokerConfig<GenericService> invokerConfig = new InvokerConfig(
            "com.dianping.demo.DemoService",
            com.dianping.pigeon.remoting.common.service.GenericService.class);
        invokerConfig.setTimeout(1000);
        invokerConfig.setGeneric(GenericType.JSON.getName());
        invokerConfig.setCallType("sync");
        /** generic call **/
        final GenericService genericService = ServiceFactory.getService(invokerConfig);
        List<String> paramTypes = new ArrayList<>();
        paramTypes.add("java.lang.String");
        List<String> paramValues = new ArrayList<>();
        paramValues.add("0000000001");
        String result = genericService.$invoke("getById", paramTypes, paramValues);
    }
}

DSL Design

The DSL describes tasks, their dependencies, inputs, and outputs. Example:

{
    "tasks": [
        {
            "url": "http://helloWorld.test.hello",
            "alias": "d1",
            "taskType": "PigeonGeneric",
            "method": "getByDoubleRequest",
            "timeout": 3000,
            "inputs": {
                "helloWorld": {"name": "csophys", "sex": "boy"},
                "name": "winnie"
            },
            "inputsExtra": {
                "helloWorld": "com.dianping.csc.pirate.remoting.pigeon.pigeon_generic_demo_service.HelloWorld",
                "name": "java.lang.String"
            }
        },
        {
            "url": "http://helloWorld.test.hello",
            "alias": "d2",
            "taskType": "PigeonGeneric",
            "method": "getByDoubleRequest",
            "timeout": 3000,
            "inputs": {"helloWorld": {"name": "csophys", "sex": "boy"}, "name": "winnie"},
            "inputsExtra": {"helloWorld": "com.dianping.csc.pirate.remoting.pigeon.pigeon_generic_demo_service.HelloWorld", "name": "java.lang.String"}
        }
    ],
    "name": "pigeonGenericUnitDemo",
    "description": "pigeon泛型调用测试",
    "outputs": {
        "d1name": "$d1.name",
        "languages": "$d2.languages",
        "language1": "$d2.languages[0]",
        "name": "csophys"
    }
}

Architecture

Pirate consists of Facade (exposed API), Parser (DSL parsing), Executor (supports Pigeon, MTThrift, HTTP, etc.), DataProcessor (post‑processing, masking), and a plug‑in system for logging and other cross‑cutting concerns.

Main Features

Decentralized design with SDK‑embedded engine.

Parallel and sequential service orchestration.

JSON DSL for easy configuration.

JSONPath for result extraction.

Built‑in and custom functions for data transformation.

Visualization of the service tree.

Support for Pigeon and MTThrift generic calls.

Tutorial Example

Goal: retrieve order status and payment status using two existing services.

{
  "tasks": [
    {
      "url": "http://test.service",
      "alias": "d1",
      "taskType": "PigeonGeneric",
      "method": "getByOrderId",
      "timeout": 3000,
      "inputs": {"orderId": "#orderId"},
      "inputsExtra": {"name": "java.lang.String"}
    },
    {
      "url": "http://test.service",
      "alias": "d2",
      "taskType": "PigeonGeneric",
      "method": "getPayStatus",
      "timeout": 3000,
      "inputs": {"paySerialNo": "$d1.paySerialNo"},
      "inputsExtra": {"time": "java.lang.String"}
    }
  ],
  "name": "test",
  "description": "组装上述接口获取订单状态和支付状态",
  "outputs": {"orderStatus": "$d1.orderStatus", "payStatus": "$d2.payStatus"}
}

Client call:

String DSL = "上述DSL文件";
String params = "{\"orderId\":\"000000001\"}";
Response resp = PirateEngine.invoke(DSL, params);

Result:

{
  "orderStatus": 1,
  "payStatus": 2
}

Impact on Development Process

By moving data preparation to the business side and using Pirate for orchestration, platform RD no longer needs to constantly verify data semantics with business RD. Both sides focus on their expertise, improving development efficiency and result accuracy.

Conclusion & Outlook

Decentralized design guarantees usability.

High service reuse and clear domain separation reduce R&D cost.

Business side can pre‑validate data, ensuring high‑quality delivery.

Unified monitoring and processing of service calls on the platform side.

Future plans include richer internal functions, support for additional protocols (e.g., HTTP), a full‑featured operation tool for DSL authoring and testing, and automatic generation of unit tests for verified DSLs.