Tencent BrowserSkill: AI Automates Your Logged-In Browser with Human-in-the-Loop

Core Principle: Local Bridge Between AI Agents and Your Browser

BrowserSkill is not a crawler or headless browser; it is a local bridge that lets Shell-capable AI agents (WorkBuddy, Cursor, Claude Code) reuse the Chrome/Edge browser you already have open and logged into. It consists of two parts: the bsk CLI local service and a Chrome/Edge browser extension. The full chain runs on 127.0.0.1: AI Agent → bsk CLI → WebSocket → browser extension → independent Agent Window that performs web actions. No credentials are uploaded automatically.

Two Key Design Differences from Traditional Automation

1. Reuse Existing Login State

BrowserSkill reads your current Chrome/Edge cookies and login sessions directly. No new browser profile, no password handling by AI. Works with WeChat Official Accounts, Taobao, Xiaohongshu, enterprise SaaS — any site where you are already logged in.

2. Human-in-the-Loop Mechanism

When CAPTCHAs, QR-code scans, or confirmation dialogs appear, the agent pauses automatically and waits for human action. After manual resolution, the task continues. No blind retries.

Window Modes

Independent Agent Window (default) — does not interfere with your browsing.

Borrow mode — temporarily reuses an existing tab, returns it after task completion.

Recording Logic vs. Coordinate Replay

BrowserSkill records page transitions, element locators, and interaction logic into trace.json — not mouse coordinates. After recording, changing target URLs or search keywords lets the AI relocate elements dynamically, unlike rigid macro playback.

Comparison with Playwright and browser-use

BrowserSkill : reuses local browser session; records element logic; native human-in-the-loop; suited for backend exports, comment research, repetitive logged-in tasks with human supervision.

Playwright : persists state via separate browser profile; code-defined selectors requiring updates on UI changes; human intervention must be coded; best for automated testing, large-scale crawling with pre-prepared accounts.

browser-use : can load real browser profile; LLM-driven real-time page understanding; supports human intervention; ideal for open-ended web exploration and uncertain page structures.

Two Core Capabilities

Capability 1: One-Shot Natural Language Tasks (No Recording Needed)

For simple, one-off collection tasks, describe goal, allowed actions, forbidden actions, and output format in a prompt. Example prompt template:

Use browser-skill to open my logged-in Taobao, search “air fryer parchment boxes”, collect top 50 products: title, price, monthly sales, shop name, shop location. Read-only, no orders, no form submits, pause on CAPTCHA. Output as Markdown table. Close session when done.

Suitable for ad-hoc competitor research, content extraction, job listing collection.

Capability 2: Operation Recording — Demonstrate Once, Reuse Forever (Core Highlight)

Complex multi-step workflows are hard to describe in text. Recording solves this:

Click BrowserSkill extension icon → open Operation Recording → enter task topic and start URL → tool generates recording prompt.

Paste prompt into WorkBuddy → agent launches recording window.

Manually perform the full workflow: click, paginate, export — no need to slow down.

Click “End Recording” → auto-generates trace.json.

Add business rules (e.g., variable page counts, element differences across items).

Future runs: just change target object; agent reuses the recorded flow.

Typical scenario: weekly batch download of product reviews on e-commerce, fixed report exports from admin panels. Record once, reuse repeatedly, drastically cutting prompt engineering cost.

Complete Installation & Verification Guide

Environment Requirements

OS: Windows / macOS / Linux

Browser: Chrome or Edge only (Firefox not supported)

Agent must support local Shell calls (guide uses WorkBuddy)

Two components: bsk CLI + BrowserSkill browser extension. Extension must be installed manually by the user ; AI cannot perform extension authorization.

Step 1: Install bsk CLI via WorkBuddy

Create a new WorkBuddy session, paste this prompt to let the agent run the install script:

Follow https://raw.githubusercontent.com/Tencent/BrowserSkill/main/AGENT_INSTALL.md to install and configure BrowserSkill locally. Requirements: 1. Install bsk CLI and browser-skill. 2. On Windows, do not use sudo. 3. Run bsk --version and output version. 4. Run bsk doctor and return full diagnostics. 5. If browser extension install is needed, open official store page and pause for me to complete manually.

Manual Fallback Commands

Windows PowerShell:

irm https://raw.githubusercontent.com/Tencent/BrowserSkill/main/install.ps1 | iex
bsk install-skill --yes
bsk doctor

macOS / Linux:

curl -fsSL https://raw.githubusercontent.com/Tencent/BrowserSkill/main/install.sh | sh
bsk install-skill --yes
bsk doctor

Step 2: Install Browser Extension & Verify Connection

Chrome Web Store:

https://chromewebstore.google.com/detail/hhcmgoofomhgciiibhipgmgkgnoenaoi

Click “Add to Chrome/Edge”. If store inaccessible, download extension zip from GitHub Releases, enable Developer mode at chrome://extensions/, load unpacked.

Open BrowserSkill extension popup, wait for connection status to turn green.

Back in WorkBuddy, send:

Extension installed and shows green, please run bsk doctor again and send full results.

Step 3: Acceptance Criteria (All Four Must Pass)

bsk --version

returns version number.

Extension popup shows green connection status. bsk doctor shows all checks ok or na, no fail.

New WorkBuddy session calls browser-skill, successfully launches independent Agent window, visits a page, returns result.

Note: After installation, you must create a new WorkBuddy session ; old sessions cannot load the newly installed Skill.

Step 4: First Test Task (Public Page, Verify End-to-End)

Paste this prompt to test read-only access:

Use browser-skill to open https://example.com. Read page title and body, summarize into 3 Chinese bullet points. Read-only, no login, no forms, no submits. Close browser-skill session after completion and tell me if task succeeded.

Success indicator: independent Agent window opens, returns summary, closes automatically.

Three Ready-to-Use Prompt Templates

Template 1: Long-Page Content Collection (Articles, Docs, Detail Pages)

Use browser-skill to open [article URL]. Read full content, scroll/expand as needed. Extract title, author, publish date, core arguments, key data, original link. Separate facts from opinions; mark unverifiable as “to be verified”. Read and screenshot only, no login, no comments. Close session when done.

Template 2: Multi-Page Comparative Analysis

Use browser-skill to open these 3 public pages sequentially: [URL1], [URL2], [URL3]. Compile a table with: page title, publish date, core conclusion, data scope, original link. If pages contradict, list each separately — do not fabricate a unified answer. Read-only, no downloads, no login. Close session when done.

Template 3: Form Filling (Draft Mode, No Submit)

Use browser-skill to open my logged-in [backend name]. Fill the following into a new draft: [content]. Allow create/edit draft, but forbid clicking publish, submit, send, or confirm buttons. After filling, screenshot and pause for my manual review. Stop immediately on login, CAPTCHA, permission requests, or any payment steps.

Frequent Troubleshooting Checklist

bsk: command not found

— PATH not configured. Fix: new WorkBuddy session, let agent check PATH; on Windows manual install, ensure bsk.exe path added to system PATH. extension connected detection fails — verify extension enabled, popup shows green; corporate network or browser security policies may block local extension communication, require admin allowlist.

WorkBuddy cannot call browser-skill — run bsk install-skill to confirm binding; create new session; prompt must start with explicit “use browser-skill”.

Wrong browser opens — run bsk browsers to list detected browsers, specify target in prompt.

CAPTCHA / 2FA appears — agent auto-pauses, human completes verification, then instruct agent to continue. Never auto-retry CAPTCHA.

Continuous clicks no response — stop task immediately, let agent re-read page state, locate blocker, avoid infinite click loops.

Security Boundaries

Permission Control: High-risk actions (publish, delete, payment, permission changes, order submit) require human confirmation; never let AI execute directly.

Account Safety: Prefer isolated browser profiles with only needed login sessions. Do not let agent extract cookies, tokens, passwords. Official policy forbids credential extraction.

Data Validation: Page structures change; human must spot-check automated results. Test batch tasks with small samples first, scale only after stability.

Source Verification: Download BrowserSkill only from official GitHub repo; avoid third-party modified builds.

Platform Compliance: Respect target site ToS; high-frequency bulk collection risks account bans.

Critical Distinction: BrowserSkill’s local communication runs on-device, but page content, screenshots, and text are sent to the upstream LLM (WorkBuddy/HY3) for analysis. Local bridging ≠ local inference. Exercise caution with sensitive internal, financial, or personal privacy pages.

Applicable Scenarios & Selection Guidance

Best Fit for BrowserSkill

Weekly fixed backend report exports, data aggregation.

E-commerce competitor review mining, bulk collection from logged-in platforms.

BOSS Zhipin / Liepin job listing batch research, candidate/position list compilation.

Self-media content research: scrape comment-section user questions for topic ideation.

Repetitive backend form filling — draft entry, human final confirm & submit.

Not Recommended

Large-scale, high-concurrency web crawling (prefer Playwright).

Complex frontend automation testing (Playwright ecosystem more mature).

Fully unattended 7×24 automation (lacks human fallback).

Conclusion

BrowserSkill’s greatest value is not merely letting AI click web pages, but solving the hardest part of AI agent deployment: connecting to the real, already-logged-in web environment you use daily .

Two usage paths: one-off research tasks via natural language; fixed repetitive workflows via one-time recording then reuse. AI handles tedious pagination, collection, form filling; humans define goals, handle exceptions, confirm critical actions, verify data. It’s a capable AI assistant, but you always retain control.

Project repository: https://github.com/Tencent/BrowserSkill