How to Build a New‑Retail Data Middle Platform with DataWorks

New Retail Business Model

To build a data middle platform, a new‑retail enterprise must first deeply understand its business. Different channels—online e‑commerce, offline stores, apps, distribution, supply chain—should be mapped to a unified commercial model before data integration.

Product Technical Architecture Design

The core technical stack relies on Alibaba Cloud infrastructure. DataWorks + MaxCompute support the data middle platform for over a decade. Data sources span structured and unstructured formats from apps, stores, IoT devices, and HR systems, feeding into a layered architecture: source data, integration, processing, and data assets.

The platform uses MaxCompute for offline warehousing, Flink for real‑time computation, and Hologres for interactive analytics. Data services are exposed via tables, APIs, or reports to downstream business applications.

DataWorks provides end‑to‑end capabilities: data integration (50+ batch sources, 10+ real‑time sources), metadata management, development tools (DataStudio, HoloStudio, StreamStudio), and open APIs for custom extensions.

Data Middle Platform Construction Steps

1. Define business goals and commercial model. 2. Design product‑level technical architecture. 3. Establish a dedicated data middle‑platform team to own data assets. 4. Build a layered data model (ODS, DWD, DWS, ADS) with clear responsibilities. 5. Implement data processing pipelines (code conversion, business judgment, joins, aggregation, filtering, conditional selection, business parsing). 6. Deploy task scheduling and monitoring in DataWorks to ensure timely, reliable data delivery.

Operations, Governance, and Security

Data quality monitoring covers table size, row count, enum changes, primary‑key conflicts, and format validation. Baseline management prioritises critical tasks (level 8) and allocates resources accordingly. DataWorks also offers intelligent alerts that predict potential baseline breaches.

Security is enforced at four levels: platform, project, table, and field. Sensitive fields (e.g., ID numbers, phone numbers) can be masked, and access requires appropriate approvals.

Value Delivered by the Data Middle Platform

The platform transforms raw data into business‑ready services, enabling predictive applications such as hourly sales forecasts for fresh‑food items, automated discount triggers, and fine‑grained inventory health monitoring, thereby turning data into a strategic asset.