Meituan Waimai Feature Archive Platform: Architecture, Tag System, and Data Processing
Meituan Waimai’s Feature Archive platform processes billions of daily orders by managing ~200 user and 400 merchant tags through a three‑layer architecture—Hive, Elasticsearch, HBase, and MySQL—offering visual tag selection, instant self‑service queries, full data extraction, and a predicate‑logic query language, while supporting future extensibility.
Background: Meituan Waimai's delivery platform has grown to over 9 million daily orders by Dec 2016, requiring efficient data‑driven operations. The Feature Archive platform provides user and merchant tag management, data query, storage, generation, and export, covering ~200 user tags and 400 merchant tags.
Core requirements: convenient visual tag selection, instant self‑service queries, and full result retrieval.
Technical challenges: processing massive data into tags, selecting appropriate storage/query engines for instant queries and full data extraction.
System architecture: three layers – data processing layer (Hive warehouse), data service layer (waimai_data_feature_service) using Elasticsearch for tag storage/query, HBase for instance storage, MySQL for metadata, exposed via Thrift APIs.
Service architecture: includes configuration (MCC), MySQL, HBase, Elasticsearch, AOP (logging, feature switches), and tag management modules (FeatureShow, GenESFilter, ParseQueryResult).
Tag system: two main categories (merchant and user tags) with dimensions and attributes; tags are stored in ES and managed via a flexible tag management framework.
Tag production pipeline: data sourced from business DBs and logs → Hive ODS → transformation → aggregation → topic tables → application buffer → final ES index rebuilt daily.
Storage & query: ES provides real‑time search and analytics; HBase handles large instance storage; MySQL stores metadata. Two ES query modes: FromToQuery (shallow pagination) and ScrollQuery (deep pagination for full data extraction).
Tag query language: a first‑order predicate logic allowing logical operators (and, or, not) and comparison operators (>, <, =, !=) on tags.
Conclusion & outlook: The platform supports marketing and operations, with future work on third‑party tags, customizable tags, and deeper data mining.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Meituan Technology Team
Over 10,000 engineers powering China’s leading lifestyle services e‑commerce platform. Supporting hundreds of millions of consumers, millions of merchants across 2,000+ industries. This is the public channel for the tech teams behind Meituan, Dianping, Meituan Waimai, Meituan Select, and related services.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.