Design and Implementation of a Traffic Control Platform for E-commerce Search and Recommendation

R&D Background

The platform aims to solve three operational demands: (1) rapid response to emergencies that cannot rely solely on algorithm learning; (2) support for new products or new users lacking data, which requires frequent manual adjustments; (3) experimental projects that need targeted traffic for small‑scale tests before full rollout.

Why Build a Platform?

Without a platform, each new strategy requires 10‑12 person‑days of development across multiple teams (BI, engine, recall, ranking, real‑time warehouse) plus additional PMO, testing, and change‑request resources. Re‑building the same logic for different scenarios also incurs redundant costs.

Solution Overview

The system is divided into five modules:

1. Traffic Operation Center : Enables operators to configure strategies, switch product sets, adjust targeting, approve changes, and view reports without waiting for development cycles.

2. Traffic Service Center : Matches operational rules with online requests, orchestrates algorithm control, recall adjustment, logging, and rate‑limiting.

3. Data Computing Center : Updates offline product and metric data to the search engine and provides minute‑level real‑time statistics for algorithm control.

4. Traffic Algorithm Control (Central) : Dynamically adjusts control intensity per item based on product features, user profile, estimated score, target progress, and growth speed. It handles smoothing, overload protection, and acts as the system’s brain.

5. Infrastructure : Provides stability, monitoring, and troubleshooting facilities.

Technical Chain

The request flow enters from various scenes, carries query or user triggers, and is matched against plans in the Traffic Service Center. Effective strategies generate weighted recall results ( <action, weight> ) that are re‑ranked upstream. Real‑time data is collected via client‑side logging and server‑side Kafka, aggregated in an ODPS table, and fed back to the control engine.

The platform supports both search‑text recall and recommendation‑type X2i recall, ensuring candidate sets remain a subset of the original pool for relevance consistency.

Real‑Time Metrics

A table of key metrics (e.g., effective control PV, plan‑level PV, scene‑level PV) is maintained with daily reset at midnight. Each metric is keyed by plan ID, scene, and experimental bucket, enabling precise exposure accounting.

Algorithmic Control Logic

For quota‑preserving goals, a PID controller computes a weight: weight = 1 + pid_score (clamped to [0.1, 10]) . The final score may be score = rs * weight + relevance or score = rs * pctr / median(pctr) * weight + relevance . For proportion‑based control, the system boosts or suppresses items based on the gap between actual and target exposure ratios.

Independent Recall Chain

Unlike traditional pipelines that filter after natural recall, the independent recall chain restricts the candidate pool to controllable items, guaranteeing that controlled items are not displaced by uncontrolled ones. This approach achieves ~99% coverage for new‑product exposure uplift.

Cross‑Scene Unified AB Experiments

Unified orthogonal AB testing across search and recommendation ensures that a user sees consistent experiment assignments regardless of entry point, facilitating joint ROI analysis.

Future Directions

Planned enhancements include expanding platform capabilities (e.g., integrating community content, supporting more recommendation scenes), improving operational tooling (debug utilities, visual config), and extending control to non‑product scenarios such as keyword promotion.