How to Build a Minute‑Level Order Fulfillment Simulation Platform with DataWorks

Background

The spot‑order business originally estimated fulfillment time using algorithmic models combined with operational configurations. Because timeliness‑strategy adjustments are frequent and order data collection cycles are long, a dedicated simulation platform is required so product teams can run experiments and obtain results quickly.

Goals

Reduce decision‑making cycle to minutes by delivering simulation results rapidly.

Provide a self‑service simulation platform to save development effort.

Standardize simulation capability to accelerate model‑optimization cycles.

Challenges

Each simulation may involve over one million orders, demanding minute‑level performance.

Platform must support multiple business types and be extensible.

Simulation must accommodate future extensions of the estimation models.

Architecture

The core functions are built on Alibaba Cloud DataWorks, a big‑data development platform that provides an offline data warehouse, distributed task execution for complex SQL at minute granularity, and unified data access across heterogeneous sources.

Core Process

Simulation Task Flow

The task is decomposed into execution nodes defined by a template. Pre‑defined instance nodes, task templates, and configuration interfaces guarantee extensibility.

Key Nodes

Control Group Collection : Captures experiment data based on arbitrary time windows and business attributes.

Estimation Mode : Supports various combinations of algorithmic models and routing‑line systems.

Result (Post) Processing : Expresses overtime‑rate differences and applies promotional or pandemic‑adjusted strategies.

Result Statistics : Generates unified evaluation metrics for both business and technical stakeholders.

Node Details

Data Collection Node

Input: task ID, payment time range, business type.

Output: partitioned control‑group data keyed by task ID.

Model Node

Input: task ID, payment event range, business type.

Output: trained estimation model partitioned by task ID.

Estimation Node

Input: task ID, control‑group data, trained model.

Output: estimated results partitioned by task ID.

After‑Process Node

Input: task ID, estimation results, post‑process type.

Output: post‑processed results partitioned by task ID.

Statistics Node

Input: task ID, post‑processed results.

Output: statistical summary partitioned by task ID.

All nodes are orchestrated by DataWorks in a distributed DAG, ensuring isolation between concurrent tasks.

Domain Model

Key entities:

Template Node : Value object composed of DataWorks‑mapped template parameters.

Task Template : Entity consisting of a template ID and a list of template nodes.

Simulation Task : Aggregate root containing task ID, business type, template reference, execution node list, status, and creator.

ER Diagram

JSON Task Template

{
    "templateId":"xh_01",
    "bizType":"xh",
    "tempOps":[
        {"node":"1003843531","type":"SCOPE","params":"orderPayTime,bizType"},
        {"node":"1003864245","type":"MODEL_SCOPE","params":"orderPayTime,bizType"},
        {"node":"1003864250","type":"MODEL"},
        {"node":"1003864248","type":"MODEL"},
        {"node":"1003864251","type":"MODEL"},
        {"node":"1003864249","type":"MODEL"},
        {"node":"1003864246","type":"MODEL"},
        {"node":"1003843532","type":"ESTIMATE"},
        {"node":"1003843533","type":"AFTER_PROCESS","params":"afterProcessType,afterProcessRatio"},
        {"node":"1003843534","type":"STATISTICS"}
    ]
}

Example Simulation Task

The task uses spot‑order data from 2021‑10‑01 to 2021‑12‑30 for training and estimates fulfillment time for 2022‑01‑01 to 2022‑01‑07, applying an 80% scaling post‑process and reporting coverage, overtime, and accuracy metrics.

{
    "emulateNo":"1000223101",
    "templateId":"xh_01",
    "bizType":"xh",
    "status":"RUNNING",
    "operator":[
        {"node":"1003843531","orderPayStartTime":"2022-01-01 00:00:00","orderPayEndTime":"2022-01-07 23:59:59","type":"SCOPE","bizType":"1"},
        {"node":"1003864245","orderPayStartTime":"2021-10-01 00:00:00","orderPayEndTime":"2022-12-30 00:00:00","type":"MODEL_SCOPE","bizType":"1"},
        {"node":"1003864250","type":"MODEL"},
        {"node":"1003864248","type":"MODEL"},
        {"node":"1003864251","type":"MODEL"},
        {"node":"1003864249","type":"MODEL"},
        {"node":"1003864246","type":"MODEL"},
        {"node":"1003843532","type":"ESTIMATE"},
        {"node":"1003843533","type":"AFTER_PROCESS","afterProcessType":"1","afterProcessRatio":"0.8"},
        {"node":"1003843534","type":"STATISTICS"}
    ],
    "creator":"[email protected]",
    "createTime":"2022-05-01 12:30:00"
}

Result Record Example

{
    "emulateNo":"1000223101",
    "res":{
        "total":"xxxxx",
        "hitRatio":"xx.xx%",
        "accuracyRatio":"xx.xx%",
        "overtimeRatio":"xx.xx%",
        "distribution":{
            "2":"xxxx","3":"xxxx","4":"xxxx","5":"xxxx","6":"xxxx","7":"xxxx","8":"xxxx"
        }
    },
    "executor":"[email protected]",
    "createTime":"2022-05-01 12:35:00"
}

Future Outlook

The supply‑chain network is being expanded with routing‑line capabilities, warehouse capacity, and shift scheduling. The same simulation framework can be reused for routing‑line timeliness estimation, requiring only data‑source integration and new template definitions, without redeploying the entire system.