How web-access Transforms AI Agents from Search to Full Browser Browsing

Claude Code and Codex excel at code generation but fall short on robust networking; developers often hit dead ends with login‑required pages such as WeChat public accounts or Xiaohongshu.

1. Skill vs MCP: Methodology Over Tools

MCP (Model Context Protocol) defines a set of tools for agents, while a Skill adds the full methodology of how to use those tools. The analogy is a screwdriver (MCP) versus a screwdriver plus a manual (Skill). web‑access follows a goal‑oriented design: understand the task, choose the shortest path, and adjust dynamically instead of a fixed search‑fetch‑output pipeline.

Three Capability Layers

Base layer : WebSearch, WebFetch, curl, Jina‑to‑Markdown; adds scheduling decisions (which tool first, fallback, switch).

Core layer : CDP browser direct connection – the hardest part.

Efficiency layer : Parallel task decomposition; sub‑agents run concurrently while the main agent aggregates results.

2. CDP: Browsing Like a Real User

CDP (Chrome DevTools Protocol) lets external programs control navigation, execute scripts, manipulate the DOM, simulate input, monitor network traffic, and capture screenshots. Using Chrome DevTools MCP + CDP gives natural login‑state carry‑over, so pages that require authentication can be accessed without re‑handling cookies or tokens.

3. Real‑World Cases

Case 1 – Competitive‑Product Research

Research Dify, Coze, FastGPT across node types, model integration, private deployment, and pricing. Traditional manual method takes 2–3 hours with three browser tabs. With web‑access the task splits into three sub‑agents, each handling one platform; overall time is comparable to a single platform, achieving roughly three‑fold efficiency gain. Sub‑agents extract detailed node specifications and model provider lists that search engines cannot provide.

Case 2 – Xiaohongshu Discussion Scraping

Xiaohongshu uses anti‑scraping and dynamic rendering, so WebFetch returns empty pages. web‑access detects failure, switches to CDP mode, opens a Chrome tab, enters the keyword, waits for rendering, extracts the result list, then navigates into each post to collect comments, leveraging the already logged‑in session for identical results to manual browsing.

Case 3 – GitHub Commit‑Based Weekly Report

Generate a technical weekly report from GitHub commit logs. Agent first tries WebFetch; when JS‑rendered content is missing, it upgrades to CDP, waits for full rendering, then extracts commit messages, reasons, and impacted modules, summarizing them in product‑friendly language.

These cases illustrate three design principles of web‑access:

Decision layer : Prioritize lightweight tools; only invoke CDP when dynamic rendering, login walls, or anti‑scraping are present.

Experience accumulation : A references/site‑patterns/ directory stores site‑specific selectors and interaction patterns, updated after each successful run.

Parallel scheduling : Sub‑tasks run concurrently, each managing its own browser tab; the main task only aggregates outputs.

4. Technical Comparison: CDP Proxy vs MCP vs Puppeteer

CDP Proxy reuses the everyday Chrome instance, automatically carries login state, and avoids instance conflicts; MCP and Puppeteer require separate instances and manual login handling. CDP Proxy is accessed via HTTP API, MCP uses its own tools, and Puppeteer offers a Node.js API. CDP Proxy suits daily scraping with login reuse; MCP fits isolated environments; Puppeteer targets automated testing.

5. Why CDP Proxy Fits Agent Networking

The key advantage is native login‑state carry‑over, which cannot be replaced by other solutions when accessing content behind authentication walls.

6. Industry Trends

Shift from search‑only to full browsing paradigms for agents.

Login‑state reuse becomes a critical capability for accessing social media, intranets, and paid content.

Skill‑level abstraction on top of MCP is the next evolution, providing decision logic for when and how to use tools.

Tool limits define the boundaries of what you can accomplish.

With 2.8 K stars in its first week, web‑access hit a real pain point. Developers using Claude Code or Codex are encouraged to try it; once experienced, the previous “good enough” networking feels merely a compromise.