WebRetriever Global Challenge Opens – $15,000 Prize for Web Agent Benchmark
Today the WebRetriever Global Challenge, co‑organized by Mingluo Technology, Peking University, and leading AI institutes, opens for individuals and teams worldwide, offering a $15,000 prize pool and inviting participants to evaluate their web agents on an 800‑site, 1,550‑task benchmark that measures both navigation success and full‑task completion.
Problem
AI agents that operate inside real browsers must complete tasks in a constantly changing web environment. Existing benchmarks rely on simulated or self‑built sites and mainly measure correct operation execution, lacking systematic measurement of end‑to‑end task success.
WebRetriever benchmark
Coverage of 800 real online websites.
1,550 cross‑industry tasks spanning technology, finance, healthcare, education, government and other domains.
All interactions occur in a genuine internet setting.
Evaluation framework (NavEval)
NavEval provides automated evaluation with human‑expert agreement of 91.2 % (previous best ≈ 81 %).
Performance gap
Even the top single model attains less than 50 % basic navigation success, and the end‑to‑end task completion rate is only around 20 % (“arrival” does not equal “completion”).
Resources
Paper: https://arxiv.org/abs/2607.06118
Dataset: https://huggingface.co/datasets/Mininglamp-2718/WebRetriever
Code and leaderboard: https://mininglamp-ai.github.io/WebRetriever
Competition homepage: https://mininglamp-ai.github.io/WebRetriever_Challenge/
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.