Turning Click‑Based Web Agents into Repeatable Scripts with Microsoft’s Open‑Source Webwright

From Clicks to Code

Traditional web agents treat the browser as a persistent workstation, where the model observes a page and decides the next click, input, or scroll. Webwright adopts an “engineer’s mindset”: the model receives a terminal and a disposable browser session, writes a Playwright script, executes it, and iterates based on logs and screenshots.

How Webwright Executes

The system consists of three components – a Runner, a Model Endpoint, and a Terminal Environment – and the harness is about 1 K lines of code. The execution loop follows four steps:

Give task : Runner passes the user task, workspace state, and recent observations to the model.

Write commands : The model outputs reasoning and shell commands, typically a Playwright script.

Run script : The environment runs the commands and returns terminal output, screenshots, logs, or errors.

Fix until complete : The model revises the script based on observations, then re‑runs it in a fresh directory and performs a self‑check.

This design pushes complex web interactions back into code, allowing date selection, pagination, filtering, and table extraction to be expressed as loops, functions, and wait conditions instead of coordinate guesses.

Benchmarks

Two core benchmarks are reported.

On Online‑Mind2Web (300 real‑world website tasks across 136 sites) Webwright paired with GPT‑5.4 achieves 86.7 % automatic evaluation accuracy within a 100‑step budget, compared with Claude Opus 4.7’s 84.7 %. GPT‑5.4 is stronger on simple and medium tasks, while Claude excels on the hard split.

On Odysseys (200 long‑chain web tasks, average description length 272.3 words) Webwright + GPT‑5.4 reaches 60.1 % success with an average of 76.1 steps, surpassing the previous SOTA Opus 4.6 at 44.5 %.

These results show that keeping the same base model but switching from “click‑coordinate” actions to “code‑controlled” actions yields a clear performance jump, indicating that the bottleneck often lies in the operation framework rather than model intelligence.

Reusable Script Assets

Webwright records the “browsing history” of a task as a script, logs, screenshots, and parameters. The cost per task for GPT‑5.4 on Online‑Mind2Web is about $2.37, which is justified by the reusable RPA‑style script output. When these scripts are packaged as a parameterized CLI tool, even smaller models can benefit.

For example, Qwen‑3.5‑9B can complete tasks on sites with five or more available tools, and with crafted reusable tools it reaches 66.2 % on the Online‑Mind2Web hard split.

Because the output is code, successful tasks become assets that can be stored in a tool library, invoked by other agents, reused by smaller models, or inspected and maintained by humans.

The project already provides plugin manifests for Claude Code and OpenAI Codex under the skills/webwright/ directory, integrating the “web‑task‑as‑program” capability into existing coding agents.

Limitations

Webwright is not a silver bullet. Scripts can become stale when web pages change, requiring validation, retirement, and update mechanisms. Determining the right script granularity is challenging: overly fine scripts fragment into many tiny tools, while overly coarse scripts only fit specific tasks. Some pages remain better served by low‑level click‑and‑type actions, especially for novel, fragile, or transient interfaces.

Thus, Webwright should be viewed as a directional signal: as models improve at code generation, web agents can evolve from mimicking human mouse movements to behaving like engineers who write and debug automation scripts.