UI-Venus-1.5: How Ant Group’s Open‑Source GUI Agent Achieves SOTA Performance

As models such as Claude and OpenAI Agent evolve, competition in the GUI‑agent field intensifies, yet three core challenges—knowledge gaps, theoretical‑only approaches, and high multi‑model coordination cost—still hinder practical deployment. Ant Group’s open‑source UI‑Venus‑1.5 proposes a breakthrough end‑to‑end solution that addresses all three.

The system’s first innovation is a mid‑stage training phase that aggregates more than 30 data sources into a 10 B‑token corpus, covering semantic perception and element localization. This corpus endows the model with native GUI capabilities, upgrading it from merely “seeing” interfaces to truly “understanding” operational logic.

To overcome the “paper‑only” limitation, UI‑Venus‑1.5 employs online reinforcement learning. By executing complete trajectories in real environments, the optimization target shifts from single‑step action accuracy to overall task completion, thereby closing the gap between offline training and online execution.

For multi‑scenario adaptation, the framework adopts a “split‑then‑merge” model‑fusion strategy. Offline RL first creates expert models for three domains—localization, mobile, and web. These experts are then combined with TIES‑Merging into a single unified model, preserving peak performance in each domain while dramatically lowering deployment cost. The action space is expanded with mouse hover, shortcut keys, and multilingual templates, and the localization module adds a “refuse” capability that returns [-1,-1] when a target element is absent, eliminating the “localization hallucination” seen in earlier models.

To support massive online training, Ant builds a unified Device‑as‑a‑Service (DaaS) layer that abstracts multiple protocols into extensible plugins. Secondary‑hash routing and coroutine‑based concurrency enable stable connection of thousands of heterogeneous devices and support over a thousand concurrent training tasks, providing a solid foundation for the model’s performance gains.

Comprehensive evaluation confirms its SOTA status. Across seven benchmarks—including VenusBench‑GD and ScreenSpot‑Pro—the 30B‑A3B version achieves leading scores of 75.0% and 69.6%. In the AndroidWorld real‑world scenario, it reaches a 77.6% success rate, surpassing same‑scale competitors. The model also deeply adapts to Chinese user habits, accurately executing commands in everyday apps such as Gaode ride‑hailing, Ctrip ticketing, and Weibo interaction.

By open‑sourcing UI‑Venus‑1.5, Ant Group not only resolves the practical bottlenecks of GUI agents but also establishes a reproducible end‑to‑end paradigm: from mid‑stage corpus construction, through domain‑specific expert training, to reinforcement‑learning‑driven real‑world capability. With support for over 40 mainstream Chinese apps, AI assistants are moving from concept to reality, accelerating the automation of digital life.