Meituan Dianping User Action System (UAS): Architecture and Implementation for Real-time User Behavior Processing

With the maturation of China's internet market, user growth has stalled and coarse‑grained growth models are no longer viable. Meituan‑Dianping therefore shifted to fine‑grained operations, requiring real‑time perception of user behavior to build instantaneous user profiles and deliver timely incentives, especially for newly activated users where incentives must be delivered within milliseconds to improve retention.

The team identified three core challenges in processing massive user behavior data: (1) non‑standard reporting formats because behaviors originate from disparate business lines and different messaging systems (Mafka/Swallow, Kafka, or raw traffic打点); (2) poor timeliness when relying on traffic‑based打点, which cannot meet millisecond‑level latency requirements; (3) diverse query needs across businesses, necessitating a system that is both unified and flexible enough to support varied aggregation logic.

To address these issues, the designers applied a 5W1H model to unify event format, abstracting user actions into who, what, when, where, why, and how. Reporting was unified by building a proprietary MAPI long‑channel from clients, guaranteeing low‑latency ingestion and subsequent standardization before emitting messages to topic‑partitioned Kafka. Service unification ensured that regardless of differing processing scales or latency requirements, external consumers interact with a single UAS interface. Architecture unification aimed to create an organic pipeline from collection, through processing and distribution, to persistence and serving, enabling the platform to act as a data middle‑tier that supports rapid, flexible business development.

The overall UAS architecture separates data sources into real‑time (client‑side打点 and backend messages via Mafka/Kafka) and offline (Hive/HDFS). Offline processing relies on MR and Spark modules, with the internal XT platform providing ETL job development and scheduling. Real‑time processing uses Storm: Spouts ingest the real‑time message queues, Bolts execute business logic, and the framework provides fault‑tolerance via heartbeat and Acker mechanisms. Near‑real‑time processing bridges Kafka to Hive with latency under 15 minutes for workloads that do not demand strict real‑time guarantees. Processed data is stored as detailed records in search clusters (with expiration policies) and aggregated summaries; for high‑performance scenarios, NoSQL stores such as Cellar (based on Tair) and Squirrel (based on Redis Cluster) are employed, choosing Cellar for low‑frequency, small‑value data and Redis for high‑concurrency, low‑latency needs, including HyperLogLog‑based statistics.

System traits emphasize flexibility through the Transformer component, which lets businesses extend event attributes without changing core code; query flexibility via service‑registered or plugin‑based logic hosted on UAS; and modularity that permits swapping technologies at each pipeline stage. Low latency is achieved via a Lambda architecture: a batch layer pre‑computes historical views, a speed layer processes real‑time increments, and a serving layer merges both. Availability is ensured by authentication, physical isolation of core behaviors, container‑based auto‑scaling (HULK), strict permission audits, and data desensitization. Comprehensive monitoring tracks processing latency, event counts, and anomalies, enabling rapid detection of regressions or abnormal behavior patterns.

Since deployment, UAS handles over 4.5 billion behavior reports per day, reduces core behavior reporting latency from seconds to milliseconds, tracks dozens of behavior types, and provides real‑time services averaging ~1.5 billion calls daily with a 3 ms average response time and a 99th‑percentile of 10 ms. These improvements have markedly enhanced user experience compared with previous T+1 latency.

Future work includes enabling configurable, user‑defined aggregation computations in the real‑time view, and enriching real‑time behavior streams with feature‑level intent prediction by combining real‑time features with historical user portraits. The author, Zhu Kai, is a senior engineer at Meituan‑Dianping with extensive backend architecture expertise and deep experience in real‑time data processing, currently leading the development of the operational‑business middle‑tier that underpins fine‑grained user operations.