Vercel's agent-browser: How AI Agents Navigate Web Pages via Numbered Text
Vercel Labs' agent-browser enables AI agents to control web pages by converting DOM into numbered text snapshots, using a CLI-daemon architecture over Chrome DevTools Protocol; analysis reveals 8.89M monthly npm downloads driven by CI/CD pinning, design choices like version-locked skills and CLI/MCP parity, but notes Windows limitations and anti-bot constraints.
Core Approach: Numbered Text Snapshots
agent-browser avoids two common paths for web agents: sending screenshots to multimodal models (token-heavy, fragile) or writing brittle CSS selectors. Instead, it renders the page into a compact, numbered text representation. Each interactive element receives a unique reference like @e1. The agent calls snapshot to get this list, then issues commands such as click @e1 and get url to verify. The browser stays alive between commands via a background daemon controlled through the Chrome DevTools Protocol; each --session spawns an independent daemon and browser instance.
CLI Workflow Example
agent-browser open example.com
agent-browser snapshot -i # interactive elements, each with @e1 ref
agent-browser click @e1 # click it
agent-browser get url # read back to confirm
agent-browser closeA snapshot output line looks like: - link "Learn more" [ref=e1] References are scoped to the snapshot; any navigation or DOM change invalidates them, forcing a fresh snapshot. This makes state expiration an explicit rule rather than a hidden cache.
Honest Error Reporting
The tool refuses to lie about success. If a target is hidden, disabled, off-screen, or blocked by a cookie banner, click returns an error immediately. This forces the agent to handle obstructions before retrying — a single design choice the author values more than adding ten new commands.
Recording and Replay
The record command captures a video of the session and extracts a timestamped contact sheet, showing exactly which region the agent clicked at, e.g., 1.474 seconds. This visual audit is far quicker than parsing logs.
Two Clever Design Decisions
Skills follow the CLI version. The skills.sh installer only adds a thin stub; actual skill implementations are fetched from the installed CLI via agent-browser skills get core. This eliminates version drift between skills and the core binary.
CLI and MCP share a single source. The project's AGENTS.md mandates that any change to CLI commands, parameters, or outputs must be mirrored in mcp.rs within the same pull request. MCP tools route through the CLI parser, preventing the common divergence between CLI and MCP interfaces.
Binary Footprint and Release Cadence
Rust binary: 5.29 MB (91.6%); TypeScript: 349 KB (6.0%). Each release publishes seven binaries covering all major platforms and architectures — not a thin Node wrapper.
Session Persistence Options
Three mechanisms reuse authentication state: state (export/import cookies and storage), auth, and --auto-connect (attach to an already-running Chrome). This eliminates repeated login flows for daily internal admin tasks.
Community Metrics: Stars vs. Commits
agent-browser: 833 commits, 119 contributors, 2.72 commits/day, 139 releases in 306 days. Its sibling agent-desktop: 335 commits, 10 contributors (one author contributed 230 commits). agent-browser shows a genuine community; agent-desktop resembles a personal project absorbed into the org. Watchers (112) are far fewer than stars (50k), indicating most stargazers never engage — commits are a harder signal.
The project carries a LABS badge: it is not a supported Vercel product and has no SLA.
Known Limitations
Windows support is weak. Official docs claim x64 support, yet 133 issues mention Windows. Issue #2369 reports a 30-second hang on first open with os error 10060. Third-party wrappers advise Windows users to use WSL.
Not an anti-crawling tool. It solves "agent cannot understand the page," not "site blocks automation." Anti-bot defenses will still block it.
--allowed-domains blocks subresources. The allowlist intercepts all subresource requests (CSS, JS, fonts from CDNs). Omitting CDN domains breaks page rendering. WebSocket interception is best-effort; the domain list is not a firewall.
First install requires a browser download. After npm i -g, you must run agent-browser install to fetch Chrome for Testing. Air-gapped or offline CI environments must bake this step into their images.
Who Should Adopt
Suitable for: Agent-driven web research, automated form filling, UI regression screenshots, reusing login state, and teams with existing Playwright scripts wanting a thin wrapper.
Unsuitable for: Deterministic CI testing (Playwright is better), heavy anti-bot scenarios, native Windows shell environments.
Author's Take
This is one of the few tools with an interface narrow enough to keep agents from getting lost, yet narrow enough that you must accept frequent breaking changes in 0.x. Teams already writing Playwright can add agent-browser as a thin layer; new teams can start directly with it, provided they pin versions. The mistake is wanting the tool but refusing to lock versions.
GitHub repository: https://github.com/vercel-labs/agent-browser
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
