Mastering Data Middle Platform: A 5‑Step Blueprint for Enterprise Success

Data middle platforms are built to serve applications, and constructing one for internal and external enterprise operations requires a mature methodology. The data middle platform construction methodology consists of five stages: high‑level planning, system design, development implementation, trial run, and continuous operation.

Data Middle Platform Construction Five‑Step Method

01 High‑Level Planning

The planning phase is split into business planning led by a business architect and data planning led by a data architect. Business inputs are gathered, and technical feasibility is assessed to produce a viable blueprint.

1. Business Planning

Business planning includes three steps: business research, blueprint design, and application design.

(1) Business Research

Business research covers two aspects:

Strategic and organizational interpretation – the enterprise strategy defines the ceiling for the data platform and its goals.

Research interviews – questionnaires or targeted interviews with business experts, supplemented by reports, system materials, etc.

(2) Blueprint Design

Based on business research, data current state, and pain points, data domains are abstracted and organized, forming a blueprint that captures both current data assets and future direction.

(3) Application Design

After confirming data feasibility, analysis scenarios and intelligent applications are visualized into PRD documents and prototypes.

2. Technical Research

Technical research surveys the enterprise’s overall IT landscape, covering core business systems and information‑security requirements.

It includes business system mapping, data flow relationships, and security policies to inform data handling processes.

3. System and Data Research

The goal is to clarify the types, distribution, storage, and management status of enterprise data resources, performed per business system.

Key aspects include identifying data sources (structured, semi‑structured, unstructured), connection details, timestamps, update markers, and volume metrics.

4. Overall Planning Output

Business‑side research yields indicators, tag lists, and data‑application plans; technical‑side research produces system‑level outputs and summary reports.

02 System Design

System design comprises overall design, data design, and platform design.

Overall Design (Data Architecture, Platform Architecture, Development Standards)

After planning, the overall architecture follows Alibaba’s OneData concept: unified data subject, unified modeling, unified services, and a comprehensive data‑management system.

Data Architecture

The data architecture builds on the requirements gathered in the research phase and includes OneModel, OneID, and OneService.

OneModel emphasizes a single processing of data to avoid redundant transformations across applications. Implementation includes:

Topic‑level management (standardized table and field naming).

Standardized data formats and field definitions.

Consistent metrics via a global data dictionary.

Layered data models: ODS, DWD, DWS, ADS/DM, DIM.

Comprehensive data coverage to treat data as an asset.

OneModel is divided into four parts: business blocks, data domains, bus matrix, and data layers.

OneID includes configuration, data processing, rule calculation, and storage/display of unified entity IDs.

OneService provides unified data services, covering service unit design, API design, API approval, and API operations (monitoring, alerts, throttling).

Platform Architecture

Based on earlier research, a macro‑level technical architecture diagram is created, covering data collection, storage, processing, and network deployment.

Data Model Design Standards

Good data models facilitate efficient organization of enterprise data assets. Standards include naming conventions, global dictionaries, and layered modeling (ODS, CDM, ADS).

03 Development Implementation

Implementation is divided into environment setup, data integration, and code development.

1. Environment Setup

Deploy big‑data clusters, data‑development platforms, and intelligent data‑application tools according to the resource plan and deployment scheme.

2. Data Integration

Define table‑level integration strategies (full load, incremental, frequency) and monitor data quality.

3. Code Development

Includes data‑model development, monitoring code, validation, application development, and performance testing.

04 Trial Run

After launch, validate metric definitions, data‑application effects, and overall accuracy through a trial period.

1) Data Migration

Full‑load migration for incremental models; direct extraction for full‑load models.

2) Data Batch Execution

Run end‑to‑end tasks, measure runtime and resource usage, and optimize.

3) Data Validation

Compare core indicators with existing system metrics and model outputs.

4) Application Validation

Verify external service interfaces with downstream applications, optimizing call frequency and timing.

Historical Data Re‑run and Testing

When business definitions change, reprocess historical data with impact assessment, backup, definition adjustment, and thorough validation.

05 Continuous Operation

After stable trial, proceed to production launch and ongoing operation.

1. Formal Launch

Two steps: cut‑over plan (if replacing legacy systems) and launch rehearsal to expose issues.

Post‑launch, establish data consistency rules, primary‑key uniqueness, and resource‑usage alerts.

2. Operational Assurance

Monitor system performance, collect product feedback, analyze application usage, and track data‑chain results to identify gaps and plan expansions.

(Source: Yunxi Technology)