Building a Marketing‑Oriented Data Middle Platform: Concepts and Practices

The presentation, titled “Data Middle Platform Construction for Marketing Scenarios,” was delivered by Wang Chen, a platform architect at Sensors Network Technology (Beijing), and organized by the DataFun community.

Sensors Data is a Chinese big‑data analysis and marketing‑technology service provider founded seven years ago, now employing about 1,200 people and serving over 2,000 customers, with extensive industry experience and a joint consumer‑behavior analysis standard with the China Academy of Information and Communications Technology.

The talk begins by describing the marketing data needs of enterprises: integrating online (APP, website, mini‑program, public account) and offline (store visits, purchases, memberships) behavior with business transaction data to build comprehensive user profiles, then applying models such as RFM and AIPL to infer user value and guide lifecycle marketing.

To avoid siloed systems, a data middle platform is proposed with four core capabilities: data integration, data modeling, data processing, and data application, each addressing specific challenges in the marketing data flow.

Data integration involves ingesting four main data sources—online/offline behavior, business systems (orders, membership, product management), third‑party advertising data, and enterprise data warehouses or lakes—through a visual data‑receiving framework that supports simple SQL‑based ingestion, SeaTunnel for complex transformations, and comprehensive monitoring including data lineage.

Data modeling focuses on turning raw data into usable structures for marketing activities, enabling rapid segmentation of customer groups (e.g., high‑frequency high‑value users) and supporting timely campaign execution without lengthy development cycles.

The platform extends the classic EUI (Event‑User‑Item) model to a multi‑entity model, allowing entities such as stores, employees, and products to have their own attributes and events, which fits well with NoSQL architectures and simplifies handling of complex relationships in retail scenarios.

Data processing and application provide a data‑asset view, detailed tag and clustering capabilities, plugin‑based label rules, and the Entity Query Language (EQL) that offers SQL‑like power with a more business‑friendly syntax; open APIs enable streaming output, subscription, and high‑concurrency queries for real‑time marketing use cases.

The Q&A section addresses technical details such as using CDC (e.g., binlog) to sync business databases like MySQL, Oracle, or PostgreSQL into the platform, and how NoSQL stores support multi‑level index queries through modeled joins and optimized computation frameworks.