The paper introduces CASCADE, a deployment‑time learning framework that lets LLM agents continuously select and reuse past cases via a contextual‑bandit approach, achieving higher long‑term success rates across diverse online tasks without updating the base model.
The article presents the LWD (Learning While Deploying) framework, detailing its reinforcement‑learning‑driven data flywheel, the DIVL value‑evaluation and QAM policy‑optimization modules, and experimental results where a dual‑arm robot improves success rates by up to 17% and reduces cycle time by 23.75 seconds across eight real‑world tasks, surpassing strong baselines.
SePT (Self‑evolving Post‑Training) shows that a large language model can improve its mathematical reasoning ability by about ten percentage points using a reward‑free online self‑training loop that decouples generation temperature from standard SFT, matching or surpassing RL‑based methods without harming general performance.
Microsoft’s Online Experiential Learning framework lets large language models continuously self‑evolve after deployment by extracting experience from user interactions and consolidating it into model parameters, eliminating the need for human labels, reward models, or server‑side environment access, and demonstrating scalable gains across tasks and model sizes.
GR4AD, Kuaishou’s generative recommendation system, redesigns the entire ad pipeline—from tokenizing multimodal ad material to value‑aware learning, lazy decoding, and dynamic beam search—delivering over 4 % revenue lift, higher eCPM, and sub‑100 ms latency for more than 400 million users.
This article explores the challenges and state of long‑term memory for AI agents, reviews mainstream industry solutions such as RAG, HRM, Titans and Engram, and proposes a four‑layer memory architecture with data acquisition, organization, utilization, and feedback loops to enable agents that remember and forget like humans.
The article reviews the 2025 evolution of Tencent advertising’s bidding system—from the second‑generation GMPC control algorithm through the third‑generation MRB reinforcement‑learning model to the fourth‑generation generative RL GRB—detailing architectural upgrades, multi‑channel modeling, training pipelines, and experimental gains, and outlines the 2026 AI‑agent roadmap.
The SOP (Scalable Online Post‑training) framework redesigns VLA post‑training from offline, single‑machine, sequential processing to a distributed, parallel online system, allowing robot fleets to continuously learn, share experiences, and scale intelligence while maintaining stability and generalization in complex real‑world environments.
This talk examines the core challenges facing AI coding agents—limited context windows, knowledge accumulation, and software‑engineering complexity—while outlining practical solutions such as context providing, RAG, fine‑tuning, online learning, feedback loops, and multi‑agent collaboration to move toward truly human‑like, continuously learning coding assistants.
This article explains the EasyRec recommendation system’s training and inference architecture, detailing optimization techniques such as embedding parallelism, CPU/GPU placement, XLA and TRT fusion, online learning pipelines, network compression, and real‑world deployment results that dramatically improve throughput and latency.
It presents concise summaries of six recent arXiv papers on unsupervised domain adaptation, efficient forecasting, SHAP explanations, text‑reinforced multimodal forecasting, online prediction with feature adjustment, zero‑shot forecasting zoo, and a new anomaly‑detection metric, highlighting methods, datasets, and results.
This article reviews the recent work by Zhou Zhihua’s team at Nanjing University on dual‑adaptivity universal algorithms for online convex optimization, introducing a meta‑expert framework, the UMA2 and UMA3 methods, and extending them to online composite optimization with strong adaptive‑regret guarantees.
This article explains the EasyRec recommendation system's training and inference architecture, details a series of optimizations for both CPU and GPU pipelines, and describes the online learning workflow that enables real‑time model updates across large‑scale e‑commerce scenarios.
This article explains EasyRec's end‑to‑end recommendation system, covering its training‑inference architecture, a series of CPU/GPU and distributed optimizations, and a real‑time online‑learning pipeline that together improve throughput, latency, and model freshness.
This article presents a comprehensive overview of EasyRec’s recommendation system architecture, detailing training and inference optimizations, embedding parallelism, CPU/GPU placement strategies, online learning pipelines, and network compression techniques that together improve scalability, latency, and cost efficiency.
This article compiles concise overviews of recent e‑commerce research, covering real‑time online learning re‑ranking models, causal inference for user growth, full‑link data lineage, TikTok's data governance and attribution solutions, Volcano Engine's metric management, AI Agent applications on 1688, and XinXuan Group's live‑stream data architecture.
iQIYI’s minute‑level online deep‑learning framework overcomes stability, timeliness, compatibility, delayed feedback, catastrophic forgetting, and i.i.d. constraints through high‑availability pipelines, TensorFlow Example serialization, rapid P2P model distribution, flexible scheduling, disaster‑recovery rollbacks, PU‑loss adjustment, and knowledge‑distillation, delivering a 6.2% revenue boost.
This article introduces LAST, a novel online learning approach that updates recommendation models instantly at serving time, addressing real‑time learning challenges, re‑ranking complexities, and demonstrating superior offline and online performance in industrial e‑commerce scenarios.
EasyRec, Alibaba Cloud’s modular recommendation framework, unifies configurable data, embedding, dense, and output layers on MaxCompute, EMR, and DLC, and speeds training with deduplication, EmbeddingParallel sharding, lock‑free hash tables, GPU embeddings, and AMX BF16, while inference benefits from operator fusion, low‑precision AVX/AMX kernels, compact caches, batch merging, and network compression, enabling real‑time online learning and delivering higher recommendation quality at lower cost in e‑commerce.
This article introduces LAST, a novel online learning method that updates ranking models instantly at serving time without waiting for user feedback, addressing the latency and stability challenges of real‑time re‑ranking in industrial recommendation pipelines and demonstrating its superiority through offline and online experiments.
This article examines how large pre‑trained models such as GPT‑4 and BERT are integrated into modern recommendation systems, detailing their advantages, implementation strategies, real‑world e‑commerce case studies, and the technical and privacy challenges that must be addressed for effective deployment.
This article details Alipay's advertising algorithm upgrades—including sample‑enhanced cold‑start mitigation, cross‑scene and user‑segmented knowledge transfer, and real‑time feature and online‑learning optimizations—that collectively lifted CTR, CPM, and overall business revenue.
This article introduces Alibaba's Alink platform and its online learning capabilities, discusses common challenges in machine‑learning pipelines, explains Alink’s algorithm‑to‑application connection, various computation modes, usage methods, and details the evolution from Model Flow 1.0 to the more versatile Model Flow 2.0, including pipeline integration, incremental training, and embedding prediction services.
This article presents an in‑depth overview of Tencent's TRS automatic hyperparameter tuning, covering background, challenges, the evolution from Bayesian optimization to evolution strategies and reinforcement learning, a systematic platform solution, real‑world deployment results, and a Q&A session.
The paper, a joint effort by Alibaba Mama and HKBU TMLR, shows that monitoring model dynamics—specifically a newly defined memorization‑discrepancy metric—can reveal hidden accumulative poisoning attacks in online advertising streams, and introduces a discrepancy‑aware correction algorithm that consistently outperforms existing defenses across benchmark datasets.
This article explains the architecture of Weibo's popular recommendation system, the role of the weidl online learning platform, and how DeepRec’s performance optimizations—such as oneDNN operator acceleration, cost‑aware scheduling, and adaptive memory allocation—significantly improve training speed, inference latency, and overall service throughput.
This article introduces Flink ML, a DataStream‑based iterative engine and machine‑learning algorithm library, covering its overview, iterative execution engine design and API, performance comparisons with Spark ML, online logistic regression and K‑Means demos, and future development roadmap.
The authors introduce AGE, an adversarial‑gradient‑driven exploration framework that injects uncertainty‑scaled perturbations into ad embeddings to approximate the downstream learning effect, combines Monte‑Carlo dropout uncertainty, a dynamic gating unit, and achieves up to 15 % offline gains and 6 % online CTR improvement over strong baselines.
The paper introduces DEFUSE and its multi‑task extension Bi‑DEFUSE, unbiased delayed‑feedback CVR models that correct label bias via rigorous importance‑sampling and a latent fake‑negative variable, achieving superior offline performance and a 2 % CVR lift in online deployment compared with existing industry baselines.
The article surveys the re‑ranking stage of modern recommendation pipelines, detailing its architecture after recall and precise ranking, and examining how shuffling and diversity improve user experience, while multi‑task fusion, context‑aware learning‑to‑rank, real‑time online learning, and traffic‑control strategies balance accuracy, efficiency, and business responsiveness.
The article details Alibaba’s end‑to‑end online‑learning pipeline for real‑time ranking in the Taobao home‑decor channel, covering UT log parsing, full‑feature extraction, ODL sample creation, xDeepCTR model training, and deployment, which yielded up to 7.8% CTR improvement and demonstrates the value of rapid model adaptation.
This article presents a three‑part technical guide covering the fundamentals of computational advertising and real‑time bidding, detailed offline pCTR model training pipelines with feature engineering, calibration and model structure improvements, and advanced online learning techniques such as parameter freezing, sample replay and knowledge distillation, all aimed at boosting CTR performance and reducing bias in large‑scale ad platforms.
This talk explains how 58.com’s massive blue‑collar recruitment platform uses reinforcement‑learning techniques—including multi‑armed bandits, contextual MAB, and linear UCB—to address cold‑start and interest‑divergence challenges, describes the system architecture, offline evaluation, online deployment, and reports an 8% uplift in new‑user conversion.
The article describes JD's end‑to‑end online learning pipeline for retail search, covering the background, system architecture, real‑time feature collection, sample stitching, Flink‑based incremental training, parameter updates, and full‑link monitoring to achieve low‑latency, high‑accuracy model serving.
This article reviews the development history, technical routes, and recent breakthroughs of pre‑ranking (coarse ranking) in large‑scale advertising systems, focusing on Alibaba's COLD (Computing‑power‑cost‑aware Online and Lightweight Deep) framework, its model design, engineering optimizations, experimental results, and future research directions.
This talk details the design, data pipeline, feature engineering, model evolution, and operational insights of the 58 Tongzhen home feed recommendation system, covering its architecture, localization strategies, recall and ranking models, online learning, and future directions for AI-driven content delivery in the down‑market.
This article details the design, data pipeline, feature engineering, model development, and iterative optimization of the 58 Tongzhen local feed recommendation system, covering business background, user profiling, recall strategies, ranking models such as XGBoost, XDeepFM, and online learning, and future directions.
Peng Mo’s talk details Tencent Kankan’s billion‑user feed architecture—layered data, recall, ranking, and exposure control—while addressing real‑time feature generation, massive concurrency, memory‑intensive caching, and fast indexing, and explains solutions such as multi‑level caches, online minute‑level model updates, Redis bloom‑filter exposure filtering, a lock‑free hash‑plus‑linked‑list index, and distributed optimizations that halve latency to under 500 ms and support auto‑scaling and cold‑start handling.
This article shares Weibo’s experience in building and evolving its recommendation algorithms, covering the recommendation scenario, machine learning workflow, feature engineering, model upgrades, large‑scale challenges, deployment via the Weiflow platform, and the capabilities of its machine‑learning infrastructure.
This article presents a comprehensive overview of the Automobile Home recommendation system, detailing its objectives, architecture, various ranking models from LR to DeepFM, online learning mechanisms, service APIs, feature engineering pipelines, model training platforms, debugging tools, and future optimization directions.
This article details the architecture, model evolution, feature processing, online learning, and future optimization plans of AutoHome's recommendation system, covering stages from resource collection to ranking, various models such as LR, XGBoost, FM, DeepFM, and operational practices like AB testing and debugging.
The article details the five‑year evolution of Autohome's recommendation system, covering its overall architecture, the progression of ranking models from LR to DeepFM and online learning, feature engineering pipelines, ranking service APIs, AB testing practices, and future optimization directions.
This article describes iQIYI’s dual‑DNN ranking architecture that combines a high‑capacity teacher network with a lightweight student network via online knowledge distillation, addressing the trade‑off between model effectiveness and inference efficiency in large‑scale recommendation systems.
During the pandemic, Tencent Cloud University and its Most Valuable Experts launched a free online course, led by ThoughtWorks' Shi Kai, to teach enterprises how to build data middle platforms with a lean data innovation system, offering live sessions, a weekly schedule, and incentives such as USB drives and Bluetooth speakers.
The article introduces iQIYI's double‑DNN ranking architecture that combines a high‑performance teacher network with a lightweight student network through online knowledge distillation, detailing the evolution of deep learning‑based ranking models, the motivation for model upgrades, training pipelines, and experimental results that demonstrate significant latency reduction and ROI improvement.
iQIYI’s dual‑DNN ranking model uses an online teacher‑student knowledge‑distillation framework where a complex teacher DNN shares representations with a lightweight student DNN, enabling end‑to‑end training, large‑scale feature crossing, and substantially higher recommendation accuracy while cutting inference latency and model size.
The article examines various techniques—including full, incremental, online, and local updates—as well as client‑side embedding refreshes to improve the real‑time performance of recommendation system models, balancing freshness with global optimality.
Alink, Alibaba’s open‑source machine‑learning platform built on Apache Flink, offers a rich library of batch and streaming algorithms, a Python API, iterative computation optimizations, and real‑world case studies, positioning it as a powerful AI solution for large‑scale, low‑latency data processing.
The interview details how Ant Financial transitioned from offline to online machine learning by adopting the Ray distributed engine, describing their open architecture, fusion computing approach, technical advantages, encountered pitfalls, and plans to open‑source the system for broader AI and big‑data use.
iQIYI’s feed recommendation system adopts an online‑learning framework that continuously trains a massive Wide‑and‑Deep DNN on billions of streaming samples, handling dynamic user interests, OOV embeddings, delayed labels, and non‑convex optimization, enabling hourly model refreshes and delivering up to 3.8 % higher consumption versus offline baselines.
The talk details iQIYI’s effect advertising system, describing its feed and in‑frame architecture, the oCPX billing model, multi‑stage recall‑ranking pipelines, real‑time feature engineering, online FM and Wide&Deep models for sparse conversion prediction, and a smart‑bidding mechanism that balances cost, quality, and volume.
This article presents a comprehensive overview of the OCPC (Optimized Cost Per Click) advertising algorithm deployed by Phoenix New Media, detailing its background, problem definition, two‑price mechanism, smart bidding, CVR estimation techniques, online learning architecture, challenges such as data sparsity and conversion delay, and future research directions.
This article reviews the current state of machine translation, explores the challenges of ambiguity, quality, and domain specificity, and presents human‑in‑the‑loop translation techniques—including attention‑enhanced models, transformer architectures, and online learning—while discussing practical productization and deployment considerations.
This article describes how Ant Financial’s AI team redesigned TensorFlow to enable elastic feature scaling, introduced a Group‑Lasso optimizer and streaming frequency filtering, compressed models by 90%, and achieved significant CTR and efficiency gains in Alipay’s online recommendation system.
Alibaba's X‑DeepLearning (XDL) is an open‑source deep‑learning framework optimized for high‑dimensional sparse data, offering industrial‑grade distributed training, built‑in CTR/recommendation algorithms, structured compression, and online learning capabilities, with benchmark results demonstrating superior scalability and performance.
This article presents a comprehensive set of innovations—including elastic feature scaling, a Group Lasso optimizer, streaming frequency filtering, and graph‑cut model compression—that transform TensorFlow for large‑scale online learning, delivering significant CTR gains and up to 90% model size reduction in Alibaba's recommendation systems.
This article traces the ten‑year evolution of Alibaba’s e‑commerce search system, detailing four major stages—from the early Pora streaming engine to dual‑link real‑time architectures, the integration of deep and reinforcement learning, and the shift to large‑scale online deep learning—while highlighting the technical drivers and future AI‑enabled search vision.
Meituan enhanced its homepage “Guess You Like” recommendation slot by modeling user‑item interactions as a Markov Decision Process and applying an improved DDPG reinforcement‑learning agent that adjusts the ranking trade‑off parameter, uses advantage‑based Q decomposition, shares actor‑critic weights, and runs in a real‑time TensorFlow pipeline, delivering consistent lifts in click‑through, dwell time, and depth.
This article outlines the background of Ele.me's recommendation business, details the evolution of its recommendation algorithms from rule‑based models to deep learning and online learning, and explains the practical implementation of real‑time data pipelines, feature engineering, model training, and deployment.
The article presents a comprehensive overview of Ele.me's food‑delivery recommendation system, detailing its business model, platform goals, unique challenges, market‑driven efficiency mechanisms, control strategies, system architecture, model evolution, and online‑learning techniques used to balance short‑term performance with long‑term ecosystem health.
Meitu’s personalized recommendation platform for the Meipai app combines offline feature engineering, near‑real‑time streaming, and online serving to recall, estimate, and rank billions of short videos using multi‑modal content features, user profiling, online learning, cold‑start bandit strategies, and multi‑objective diversity optimization, delivering timely, diverse feeds across live, homepage, and video‑detail scenarios.
This paper introduces the SCW-LOL algorithm, a local online learning method based on Soft Confidence Weighted that extends a global model with multiple local classifiers, uses online K‑Means for sample assignment, provides theoretical error bounds, and demonstrates superior performance on ten benchmark datasets, especially for multi‑class classification.
This article describes TensorFlowRS, an Alibaba‑built extension of TensorFlow that tackles the massive compute and sparse‑feature challenges of search, advertising and recommendation by redesigning the parameter server, adding fail‑over, gradient‑compensation, online‑learning support, advanced training modes and visualisation, achieving up to 100× training speedup and improved model quality.
The O2O Machine Learning Applications Seminar, featuring experts from Meituan‑Dianping and Alibaba, explores real‑world ML implementations for online‑to‑offline services, including online learning for search, Alibaba’s Ali Xiaomi intelligent assistant, deep‑learning‑driven recommendation systems, and advertising algorithms such as CTR and CVR optimization, sharing practical insights and best practices.
In a Q&A session, a Tencent social ads data mining expert addressed competition participants' questions on data delays, full‑set versus sliding‑window statistics, dataset authenticity, Bayesian smoothing, feature selection, handling missing values, large‑scale training, feature interactions, model stacking, online mini‑batch training, and provided reference resources.
This article presents practical strategies for processing massive competition datasets—including down‑sampling, streaming feature extraction, external‑memory learning, and tool recommendations—to overcome memory constraints and improve model building efficiency.
Alibaba’s e‑commerce search engine evolved over three years from offline batch models to a sophisticated AI-driven system that integrates real‑time feature ingestion, online learning, deep and reinforcement learning, enabling dynamic personalization and decision‑making that boosts conversion during high‑traffic events like Double 11.
This article examines the challenges of high‑dimensional sparse features in search ranking, explains why traditional linear models struggle, and describes how deep neural networks with novel encoding schemes and online updates can dramatically improve CTR prediction and real‑time performance.
The article introduces Pora, a real‑time offline‑realtime analytics system for personalized search that combines high‑throughput stream processing, low‑latency computation, online learning algorithms, and a modular architecture to support continuous 24/7 operation and large‑scale performance optimizations.
This article explains Alibaba’s dual‑link real‑time computing framework, detailing its micro‑ and macro‑level pipelines, key components such as Pora, iGraph and SP, online learning architectures, pointwise and pairwise ranking models, bandit‑based strategy optimization, PID‑controlled traffic balancing, and the impressive performance gains achieved during the Double 11 shopping festival.
The article presents Alibaba's real‑time computing platform for search, detailing its micro‑ and macro‑level architectures, online learning frameworks, point‑wise and pair‑wise ranking models, bandit‑based strategy optimization, and PID‑controlled traffic regulation, and reports significant performance gains during the Double‑11 shopping festival.
This article details Meituan's end‑to‑end recommendation system, covering its four‑layer architecture, data sources, candidate‑generation strategies, fusion methods, and both linear and non‑linear re‑ranking models, while highlighting practical optimizations like AB testing and online learning.
This article outlines Meituan’s end‑to‑end recommendation system, describing its data layer, candidate‑generation triggers, fusion strategies, and machine‑learning‑based ranking models—including collaborative filtering, location‑based, query‑based, graph‑based methods, and both linear and non‑linear models—while highlighting practical optimizations such as AB testing, real‑time behavior handling, and fallback strategies.
Baidu's Big Data Machine Learning team, led by Xia Fen, unveiled a suite of five novel algorithms that together allow trillion‑scale feature processing, minute‑level model updates, and up to thousand‑fold efficiency gains in training and inference, dramatically surpassing existing solutions such as Google's billion‑feature systems.
