Model Evolution and Optimization for Recommendation Systems in a Mid‑size E‑commerce App

The Province Money Fast Report app is a mid‑size e‑commerce platform whose rapid growth in the past two years has been driven by AI‑powered recommendation services across the home feed, item detail, collection, and personal center.

Business background : The recommendation flow follows a multi‑stage conversion funnel (browse → click → coupon → third‑party checkout), requiring optimization of CTR, CVR, and GMV at each stage.

System architecture : Offline pipelines collect exposure logs and service feature logs, perform stitching and cleaning, and generate label‑matched data using a unique traceID that links request, exposure, click, and order events.

Model training and evaluation : A lightweight TensorFlow‑based training system supports configuration‑driven training, evaluation, and online deployment. Incremental training (base model on the past 90 days, daily updates) reduces training time to about one hour and improves AUC by ~0.01.

Keep online and offline feature extraction consistent by directly dumping features online.

Split training and testing data strictly by time to avoid leakage.

Shuffle data by day rather than globally for a few‑point AUC gain.

Evaluate primarily with AUC and user‑level AUC.

Model serving : Deep models are served with TensorFlow Serving, while FM models use a custom Java micro‑service. TF‑Serving provides version control, hot updates, high concurrency, and REST/gRPC interfaces.

Model evolution :

Base model: FM (Factorization Machines) replaces LR/GBDT due to its second‑order feature interactions and proven industry performance.

FM → DeepFM: Adding a shared embedding layer and a few dense layers to capture higher‑order interactions. In this specific data setup, DeepFM did not outperform FM.

DIN/DIEN: Introduced attention (DIN) and RNN‑based sequence modeling (DIEN) on user click sequences. Limited gains due to short sequences and data sparsity.

DCN and xDeepFM: Explored cross‑network (DCN) and compressed interaction network (xDeepFM) for higher‑order feature crossing; xDeepFM showed marginal AUC improvement but higher latency.

ESMM: Multi‑task model jointly learning CTR and CVR; did not surpass separate models because of task imbalance and lack of loss weighting.

Custom network: Iterative architecture refinements and hyper‑parameter tuning (hidden size, learning rate, batch size, dropout, L1/L2) yielded an additional ~2 AUC points.

Hyper‑parameter tuning : Performed staged tuning (hidden size → learning rate → batch size → regularization). Larger embedding dimensions and appropriate hash space reduced model size while preserving accuracy.

Feature processing : ID and categorical features are hashed and embedded; numeric features are bucketed and embedded. Missing values are filled with defaults. Feature selection follows coverage, accuracy, and acquisition cost analysis, with incremental addition to control complexity.

Position bias handling : Position features should be modeled separately from other features to avoid cross‑interaction, following best practices from Google’s ML engineering guidelines.

Online feature construction : Two options exist—embedding feature extraction inside TF‑Serving via custom ops, or a separate feature service. The team chose the latter to decouple feature computation from model inference, enabling caching and independent scaling.

Model serving workflow : TF‑Serving containers are wrapped with Consul for service registration. Model updates are performed one instance at a time to avoid downtime, with registration/deregistration handled automatically.

Conclusions : Deep learning models can significantly improve recommendation performance, but success depends on solid data pipelines, careful feature engineering, appropriate model selection for each business stage, and robust serving infrastructure. Continuous experimentation and alignment with business metrics are essential.

序列：[S:12312,S:2345235,S:345345,S:-1,S:-1,S:-1,S:-1,S:-1,...]
S:-1 -Hash-> 325344354 -embeding_lookup-> [.3,.5,.6,.7...]
S:-1 -Hash-> 325344354 -embeding_lookup-> [.3,.5,.6,.7...] -Mask-> [.0,.0,.0,.0....]