Can Real‑Time Learning at Serving Time Transform Recommendation Re‑ranking?

Background: Real‑time Online Learning and Re‑ranking

In industrial recommendation pipelines the re‑ranking stage refines a small set of candidates (hundreds) after recall, coarse‑ranking and fine‑ranking. Classical online learning updates the model only after user feedback is collected, which in e‑commerce can take hours or days, limiting the ability to adapt to rapid distribution shifts such as seasonal trends, events, or live‑stream commerce.

Real‑time online learning continuously incorporates newly arrived samples so the model can quickly track these shifts. It faces three challenges:

Feedback latency – user actions may be delayed for hours.

Engineering overhead – a real‑time data‑streaming infrastructure is required.

Model stability – frequent updates can destabilize the serving model.

Re‑ranking Definition and Problem Formulation

Given three candidate items and a desired output length of two (the “3‑2” case), there are six possible permutations. Each permutation yields a feedback signal from user, platform and merchant. The objective is to select the permutation that maximizes a reward function reflecting this feedback.

The reward can be expressed as: argmax_{π∈Π} R(π; u, c) where Π is the set of feasible permutations, u the user, c the context, and R the reward that aggregates click‑through, dwell time, merchant goals, etc.

Learning at Serving Time (LAST)

LAST removes the need to wait for explicit user feedback. When a request arrives, the system computes a request‑specific parameter offset Δθ for the deployed model θ₂ that maximizes the evaluator’s score for that request. The offset is applied only for the current inference and discarded afterwards, leaving the deployed model unchanged.

LAST Architecture

Traditional online learning waits for a batch of feedback samples before updating θ. LAST instead solves an optimization problem at serving time:

Δθ = argmax_{Δ}  Eval(θ₂ + Δ; u₀, c₀)

where Eval is a pre‑trained evaluator (or simulator) that predicts the reward for a given sequence. The optimization is local to the current user u₀ and context c₀, unlike the global update of θ.

Algorithm Variants

Serial version: Generate a sequence with a generator, evaluate it, adjust the generation probability based on the evaluator score, and repeat until a satisfactory sequence is found.

Parallel version: Sample multiple Δθ candidates in parallel, generate several sequences, evaluate each, and output the sequence with the highest evaluator score.

Experimental Evaluation

Offline metrics: Using NDCG and evaluator‑based scores, LAST outperformed baseline generator‑evaluator pipelines, achieving more than a 1‑point gain in evaluator score over the strongest baseline.

Online A/B test: Deployed in Taobao’s “follow” feed, LAST increased transaction volume by two points while keeping click volume stable.

Key Advantages of LAST

Updates are performed instantly at request time, eliminating feedback latency.

Optimization is request‑specific, enabling fine‑grained “per‑user‑per‑request” improvements.

The offset Δθ is discarded after serving, so the deployed model remains unchanged and the method can be plugged in without additional engineering support.

No reliance on real‑time data pipelines; the evaluator provides surrogate feedback.

LAST was developed jointly by Alibaba and Renmin University of China.