Adaptive Degradation and Recovery for JD Alliance Recommendation System During High‑Volume Promotions

During JD's major sales events such as the 618 promotion, the Alliance marketing platform experiences explosive traffic growth that challenges system stability and recommendation quality.

The recommendation system faces four main challenges: unpredictable traffic fluctuations, highly uneven traffic distribution, numerous external media scenarios, and bursty red‑packet activities that cause rapid traffic spikes and drops.

Key obstacles include difficulty estimating traffic changes, diverse recommendation strategies across scenarios, and the need for second‑level response and control.

To address these, an adaptive degradation capability is designed with five characteristics: differentiated control of scenario links, fully automated degradation and recovery without human intervention, real‑time traffic perception and dynamic adjustment, automatic restoration to full recommendation after traffic peaks, and minimization of recommendation loss through precise degradation.

Implementation details involve configuring timeout thresholds per scenario, running guardian coroutines to collect real‑time latency and timeout rates, and applying Wilson confidence interval correction (P = current timeout rate, WilsonP = corrected rate, z = 1.96) to reduce statistical error during low‑traffic periods.

Scenario‑aware control is achieved by collecting per‑scenario latency data and applying configured thresholds to enforce differentiated degradation.

Traffic is split into primary and degraded flows; only the degraded portion is subjected to degradation, with decisions based on user activity and value tags.

A linear programming model maximizes business benefit under latency constraints, selecting the optimal combination of recall, coarse‑ranking, fine‑ranking, and re‑ranking modules (objective: maximize revenue; constraints: module latency ≤ thresholds).

After solving for the optimal binary decision vector W, a call‑chain generator creates the actual pipeline, which the pipeline scheduler then executes to complete the recommendation process.

During degradation, a small portion of traffic is periodically extracted for rebound testing; successful tests trigger a stepwise expansion of traffic until full recommendation is restored.

The adaptive degradation module provides business‑agnostic APIs for profit and latency input, timeout configuration, and degradation status queries, enabling low‑cost migration to other services.

Field results show over 90% reduction in traffic loss, second‑level adaptive degradation, minute‑level automatic recovery, zero manual intervention, and no incidents during the promotion.