AMemGym introduces an on‑policy, interactive benchmark that evaluates and trains AI assistants' long‑term memory by structuring state evolution, diagnosing memory failures, and enabling agents to self‑evolve, revealing that selective memory writing outperforms passive approaches across various LLM and agent architectures.
The article analyzes the shortcomings of current click‑through‑rate benchmarks, explains why leaderboards are valuable, and proposes concrete criteria—including online evaluation, sequential test data, leakage prevention, and read‑only submissions—to build a more realistic and robust CTR benchmarking platform.
This talk explains how 58.com tackles the cold‑start and interest‑divergence problems of its massive blue‑collar job recruitment platform by modeling the recommendation process as a reinforcement‑learning task, detailing the use of multi‑armed bandit, contextual bandit, and linear‑UCB algorithms, offline evaluation pipelines, online deployment, and observed performance gains.
