Baobao Algorithm Notes

Mar 18, 2024 · Industry Insights

Inside the 2024 KDD Cup ShopBench Challenge: Tasks, Data, and Evaluation Metrics

The 2024 KDD Cup introduces the ShopBench benchmark, a large‑scale LLM competition that simulates real‑world online shopping with 57 tasks, over 20,000 questions, and multiple tracks covering concept understanding, knowledge reasoning, user‑behavior alignment, multilingual ability, and an all‑round track, all evaluated with task‑specific metrics and a hidden test set.