Design and Implementation of a Real‑Time Advertising Index System at Meituan‑Dianping

Online advertising is a common monetization method in the Internet industry. From an engineering perspective, the structure and implementation of the ad index directly determine the performance of the whole system. This article uses Meituan‑Dianping’s search advertising system as a case study to explore the engineering secrets of ad systems.

Background and Domain Requirements

The ad index must have two basic characteristics: a hierarchical index structure and real‑time index updates.

Hierarchical index structure

Real‑time index updates

Hierarchical Delivery Model

The advertising system can be abstracted as a hierarchical model consisting of Advertiser → Campaign → AdGroup → Creative. Each level controls various fields such as budget, region, bid, and targeting conditions. The hierarchical relationship is one‑to‑many.

Real‑time Update Mechanism

Fields such as audit status and budget need to be updated within milliseconds. If a change (e.g., an AdGroup becoming paused) is not reflected promptly in the index, massive invalid traffic will occur.

Industry Survey

Most open‑source index systems (e.g., Apache Lucene, Sphinx) are designed for general search engines and cannot simultaneously satisfy hierarchical structure and real‑time updates. Therefore, the ad industry either customizes open‑source solutions or builds proprietary systems from scratch.

Index Design Overview

The design focuses on stability, scalability, and high performance. The system is implemented in C++11 to avoid JVM GC pauses, follows Google C++ Style, and adopts lock‑free structures for read‑heavy workloads.

Layered Architecture

The index library is divided into three layers:

Interface layer – provides APIs for search, update, and filtering.

Capability layer – implements forward (table) and inverted indexes, allocation strategies, and extensible index types.

Storage layer – manages memory allocation, persistence, and mmap‑based file mapping.

Storage Layer

Memory is allocated via custom allocators. Two main strategies are presented: LinearAllocator: allocates a large contiguous memory block; new mmap mappings cause page‑table reloads and performance jitter. SegmentAllocator: allocates memory in fixed‑size segments (default 64 MB) to reduce jitter during expansion.

More advanced allocators include: PageAllocator: buddy‑system page allocation (4 KB pages) built on top of SegmentAllocator. SlabAllocator: object‑caching allocator built on PageAllocator to eliminate external fragmentation.

Capability Layer

Three indexer implementations are provided: NoPayloadIndexer: simple inverted index storing only docIDs. DefaultPayloadIndexer: stores additional payload (e.g., POI quality score, max bid) per keyword. GEOHashIndexer: geolocation‑based hash index.

Both forward and inverted indexes share a common dictionary file ( term) and posting file ( posting). The posting file stores docID lists and optional payloads. Query syntax is generated by GNU Bison; an example query is query=(A&B|C|D)!E. The filter syntax uses semicolon‑separated key‑value pairs.

Interface Layer

APIs include: Search: returns a filtered ResultSet (combines DoSearch and DoFilter). DoSearch: raw result set without filtering. DoFilter: applies forward‑index filters to a raw result set.

Update APIs ( Add, Update, Delete) modify both the table and the index, enabling real‑time update streams.

Performance and Throughput

The system is IO‑intensive due to many external data sources. Batch processing is used: workers aggregate queries to the same data source, reducing latency and improving throughput. The architecture also adopts asynchronous server design to further boost performance.

Production Practice and Future Plans

The real‑time ad index has been deployed in Meituan‑Dianping’s search advertising and is applicable to other non‑search scenarios. Future work includes JNI integration for Java caching, SQL support with JDBC, and additional feature extensions.

Recruitment

The article concludes with a call for engineers interested in Linux backend development, high‑performance computing, and distributed systems to contact [email protected].