Cua: Full-Stack Open-Source Computer-Use Platform with Driver, Cloud Desktop, VM, Model & Benchmark

Project Overview

Cua is a full-stack open-source foundation for computer-use agents: it gives an AI agent a real, usable computer—cross-platform desktop operation, isolated cloud desktops, local virtual machines, a dedicated small decision model, and a benchmark for evaluation and training-data export. The project positions itself as "Scale computer-use 2.0," upgrading agents from web-only clicking to freely switching between code, APIs, and graphical interfaces. It is a monorepo (main languages Rust + Python) with five independent yet composable modules, MIT-licensed, with 26,879 stars and 1,870 forks at the time of writing.

Core Highlights

1. Rare Full-Stack Completeness

The five modules are:

Cua Driver (execution layer)

Cua Fleets (cloud desktops)

Lume (local VMs)

CUA-S1 (decision model)

Cua Bench (evaluation benchmark)

Unlike most projects that only provide a driver, a benchmark, or a sandbox, Cua delivers training, evaluation, and data generation in one pipeline, eliminating the need to cobble together infrastructure.

2. Background Driver Delivery Without Stealing Your Mouse

Cua Driver lets agents operate native desktop apps and browsers on macOS, Windows, and Linux. It offers three integration paths: CLI, MCP (Model Context Protocol), and typed SDK—compatible with Claude Code, Codex, Cursor, etc. The key detail is background delivery : the agent works in the background without grabbing your mouse focus (subject to platform and application support; see official platform-support docs).

3. Zero-Friction Bench Start + MIT Free Commercial Use

Cua Bench fills the often-neglected evaluation gap: task construction, agent assessment, and trajectory export for training data. Starter tasks require no VM, Docker, or model API key —just install cua-bench[browser] and run a simulated task. The entire stack is MIT-licensed, free for commercial use, with a solid Rust/Python codebase.

Core Principles & Architectural Differences

Old Approach Defects

Every link in the computer-use chain has scattered open-source pieces, but they come from different projects with incompatible interfaces. Environment isolation relies on manual VM setup; evaluation uses custom scripts. Stitching them together is more exhausting than building from scratch.

Cua's Four-Layer Architecture

The architecture clarifies the stack in four layers:

1. Execution Layer: Cua Driver

A unified operational primitive layer that abstracts clicking elements, inputting text, and reading state into a cross-platform API. Because it runs at the system level, it supports multi-session concurrency—the official demo shows two Driver sessions simultaneously selecting cells in LibreOffice Calc and objects in Inkscape while the terminal stays in the foreground. This is the foundation of background delivery.

2. Environment Layer: Cua Fleets + Lume

Fleets provisions isolated Linux cloud desktops on demand at run.cua.ai. Lume uses Apple's official Virtualization.Framework to create local macOS/Linux VMs on Apple Silicon. Both share the same Sandbox SDK , so code written once runs locally or in the cloud—environment provisioning is standardized.

3. Decision Layer: CUA-S1

A family of small, specialized decision models. The team analogizes them to "System 1" (fast thinking): they handle quick, bounded decisions like "which field should this value go into?" or "should I interact with this element?" instead of generating long token sequences. Offloading high-frequency micro-decisions from the main model directly reduces latency and cost.

4. Evaluation Layer: Cua Bench

Integrates task definition, evaluators, and trajectory export. It deliberately achieves zero-dependency startup, turning evaluation from a researcher's luxury into a standard part of the development workflow.

The architectural judgment is clear: the computer-use bottleneck isn't that a model isn't smart enough, but that the execution, environment, decision, and evaluation layers each miss a piece . Cua fills all four at once.

Quick Start

Core workflow:

Install Driver (macOS/Linux one-liner, Windows via PowerShell):

macOS / Linux: curl -fsSL https://cua.ai/driver/install.sh | bash Windows: irm https://cua.ai/driver/install.ps1 | iex Verify first example : have the agent compute 6 × 7 in the calculator and confirm the UI shows 42.

Try Bench (zero dependencies): pip install "cua-bench[browser]", run a simulated task; evaluator reports reward 1.0 when ready.

Need a sandbox? Apple Silicon users launch local macOS VMs with Lume; for cloud isolated desktops, connect to Fleets.

Note: The above follows official documentation; platform-specific permission details are per the official platform-support docs and are not repeated here.

Team Adoption Guide

1. Team Transformation Approach

Cua fits at the GUI agent R&D foundation layer: execution via Driver, environment via Fleets/Lume, evaluation via Bench. Each layer can be adopted independently or swapped as needed —you can just add Driver to an existing agent, or replace your entire homegrown infrastructure. Boundaries are clean: it doesn't lock you into a specific model or cloud vendor (except Fleets).

2. Deployment Strategy

Driver installs via CLI scripts + MCP configuration, integrable into standard dev-machine initialization. After unifying local/cloud environment definitions with the Sandbox SDK, you get environment-as-code : VM templates and cloud desktop pool configs become repository assets, so new hires get a working environment with one command.

3. Business System Integration

Driver's typed SDK and MCP interface naturally fit into CI: UI regression tasks run by agents in Lume VMs, combined with Bench evaluators for assertions and scoring; trajectory exports feed directly into model-training pipelines.

4. Team Standards Customization

Bench's task definitions and evaluators become the carrier of team standards: encode what counts as a successful operation and which actions are out-of-bounds as tasks and evaluators. Run the benchmark before every agent release, using scores instead of gut feel for gatekeeping .

Real-World Applicable Scenarios

Teams building computer-use agents : need open-source driver for desktop operation + isolated sandbox environments.

Researchers training/evaluating GUI agents : Bench + trajectory export cover evaluation and data generation.

Developers building custom computer-operating agent foundations : pick any of the five layers, avoid piecemeal integration.

AI teaching & demos : zero-dependency Bench starter tasks suit quick classroom demonstrations.

Not suitable for : casual users wanting a ready-to-use GUI agent, or people with zero programming background—the monorepo has a learning curve.

Objective Pros, Cons & Pitfalls

Core strengths : rare full-stack completeness; Rust/Python primary + MIT license; active maintenance.

Limitations :

Complex monorepo structure : five sub-projects in one repo spanning Rust, Python, TypeScript, Go, Swift; GitHub even shows HTML as primary language due to docs volume. Extracting just the driver still requires wading through documentation.

Some capabilities tied to commercial service : Fleets cloud desktops depend on run.cua.ai, and capacity may remain billable after claim ends— official tutorials stress step-by-step cleanup to avoid surprise bills .

Local permission hurdles : Driver requires macOS accessibility permissions etc.; not plug-and-play.

CUA-S1 still early : source-only research release; model and dataset cards must be checked for applicable scope and restrictions; don't expect out-of-the-box production readiness.

Common deployment pitfalls : the entire computer-use direction is early; general GUI agent reliability is unproven, and Cua hasn't escaped that phase. Recommendation: start with Bench zero-dependency tasks and Driver single-machine commands , validate value before investing in Fleets cloud resources.

Final Thoughts

In the current computer-use wave, everyone is racing to make agents look at screens and click mice. Cua's answer is to assemble the execution, environment, decision, and evaluation infrastructure layers all at once and open-source them. It's a full suite with a learning curve, and some capabilities still hook into its cloud service—but for teams with a clear computer-use direction, the saved infrastructure time is real money.

Open-source repository:

https://github.com/trycua/cua