Agent Reach: Unlock the Entire Internet for Your AI Agent (100k Stars)

Problem: Agents Cannot Reach Most of the Internet

AI agents with built-in browsing capabilities only cover the small slice of the internet that is public, static, and has no login walls. When asked to search Xiaohongshu, Bilibili, Xueqiu, BOSS Zhipin, or Reddit, they either hit login walls, receive empty page shells, or find nothing because general search engines poorly index these Chinese platforms. Video content (Bilibili subtitles, YouTube transcripts, Xiaoyuzhou podcasts) and clean structured data (Xueqiu quotes, V2EX threads, GitHub repos) are also inaccessible.

Solution: Agent Reach CLI

Agent Reach is an ecosystem/platform project: a single Python CLI ( agent-reach) plus a SKILL.md that any agent (Claude Code, Codex, OpenClaw, Cursor, Cline, Warp, Zed) can read to discover available channels. It provides read-only access—no API keys, no account registration, no posting/commenting/liking.

Three Usage Patterns

agent-reach channels                     # list channels and their actions
agent-reach bilibili search "LLM"        # channel + action + params
agent-reach read https://www.v2ex.com/t/123  # paste URL, auto-detect channel

What It Solves: Four Coverage Gaps

Login-required features: Xiaohongshu note search, X account timelines, BOSS job details—impossible without authentication.

Video/audio content: Bilibili/YouTube subtitles, YouTube video and Xiaoyuzhou podcast speech-to-text—no corresponding fields in page HTML.

Chinese platforms: Bilibili, Xiaohongshu, V2EX, Xueqiu—general search engines cover them poorly, especially internal content.

Clean structured data: Xueqiu quotes, V2EX posts/replies, GitHub repos/issues—raw scraping returns full HTML with navbars and ads; Agent Reach returns normalized fields.

Channel Inventory: 14 Channels in Three Tiers

This tiered design is the project's most pragmatic feature.

Default (5): Bilibili, V2EX, GitHub, YouTube, Web. Work out of the box, no login needed (except Bilibili subtitles).

One-command add (6): Xiaohongshu, Xueqiu, BOSS Zhipin, LinkedIn, Reddit, Xiaoyuzhou.

Experimental (3): Twitter/X, Facebook, Instagram. Aggressive anti-scraping; may break anytime.

Codebase: runtime ~219 KB, channels ~94 KB, tests ~220 KB across 27 files—test-to-production ratio near 1:1, necessary for a project that constantly battles platform changes.

Three-Layer Architecture: The Core Is Layer Three

Agent layer: Reads SKILL.md to decide which channel to use.

Unified entry: All calls follow agent-reach <channel> <action>. Parameters are validated, then always output a single JSON with schema:

{schema_version, ok, channel, action, backend, items, meta, error}

Agents only check ok and error.code —never parse upstream tools' disparate formats.

Backend fallback lists (core): Every channel action is an ordered list of backends. Example: read Bilibili video → first try bili-cli; if it fails, fall back to Bilibili public API. Read webpage → first Jina Reader, then Exa. This turns "swap tool" into "edit a list"; agent usage stays unchanged when a backend breaks.

Error Codes Are for Agents, Not Humans

Eight error codes, each with a prescribed agent action: NEED SETUP – missing dependency/key; install per instructions. NEED LOGIN – keep existing results, ask user to log in. UPSTREAM BROKEN – platform temporarily down; tell user honestly. TIMEOUT – inspect failure stage; don't blindly retry. RATE LIMITED – wait retry_after; don't switch backends and hammer the same platform. PLATFORM BLOCKED – stop; let user verify on official page. NOT FOUND – verify the link. BAD INPUT – check channel usage.

Critical rules encoded in constants ( FINAL): NOT FOUND, BAD INPUT, PLATFORM BLOCKED, RATE LIMITED, NEED LOGIN, BUSY are terminal —no further backend fallback. This prevents blind retries that turn rate limits into bans.

Complete Walkthrough: Search Xiaohongshu & Bilibili for a New Model

Install: Send agent the install URL or run:

uv tool install https://github.com/Panniantong/Agent-Reach/releases/latest/download/agent-reach.zip
agent-reach setup

setup

installs 5 default channels plus their upstream tools (uv, ffmpeg, Node) without sudo. Only login requires human action.

Add Xiaohongshu: Tell agent "add Xiaohongshu". It runs agent-reach add xiaohongshu, installs deps, then prompts for login.

Query:

agent-reach xiaohongshu search "keyword" --limit 10
agent-reach bilibili search "keyword"

Returns unified JSON; each items entry has title, url, author, published_at, text.

Health check: agent-reach doctor --text reports "need login", "need install", "broken", or "regressed" (was good last check, now broken)—designed to catch silent platform redesigns.

Browser Automation: Thoughtful Details

Uses a dedicated headless browser (not your Chrome). Idle 10 minutes → auto-close; next call auto-opens.

First use of a site: if you have the project's Chrome extension, it copies only that site's login state—original browser untouched, no password vault access. Re-login only when session expires. Explicit policy: never log in for you, never ask for passwords.

Concurrency limits: max 2 simultaneous queries per platform, 4 total across all platforms. Queue waits 2 minutes then returns BUSY. Code comment: "same account, keep risk control calm"—a number earned from real ban experience.

Is It Worth It?

Yes. If you use Claude Code, Codex, OpenClaw, etc., and frequently need Chinese-platform content (Xiaohongshu, Bilibili, Xueqiu, V2EX), the time saved is tangible. Five channels work immediately after one install.

Technical foundation is stronger than peers. Unified error codes, backend fallbacks, real health checks, 1:1 test ratio—all built to survive a domain where failure is the norm.

But you must know three things:

Xiaohongshu ban risk is real. An issue titled "Xiaohongshu ban warning came fast, only used 2 days" includes a platform warning screenshot. User reported <15 calls. The channel will likely be blocked soon. Use a burner account; project itself recommends a secondary account.

3 of 14 channels are experimental and may die anytime. Twitter channel once lost its upstream tool when the tool's repo was deleted (likely banned). The author rewrote it. You buy "works now", not "works forever".

Project honestly states its boundaries. Docs say: "Declaring capability does not mean all networks, accounts, and content have been tested." v1.6 passed 28/40 actions; remaining 12 are "maintainer-approved for release" including Reddit (test network blocked), Instagram (rate limited), Bilibili subtitles, LinkedIn, Xueqiu hot posts/stocks, Xiaoyuzhou. A project that writes this in docs is more trustworthy than one claiming "full support".

Closing Notes

10.5k stars, 9.3k forks, 366 commits, one primary maintainer, eight contributors. CHANGELOG reveals v1.6 was a rewrite: old design let agents call upstream tools directly, each tool failed differently, agents could only guess. New version internalized that layer, enabling unified error codes. This is battle-tested, not designed in a vacuum.

If your agent often gets stuck at "I can't open this website", give it a try. Install, run agent-reach channels to see what's available—don't add everything at once; the default five last a long time. https://github.com/Panniantong/Agent-Reach MIT license, Python 3.10+. Note: the PyPI package named agent-reach is not this project; install only from the GitHub URL above.