How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt

Finding rare second‑hand gadgets or collectibles on Xianyu often requires endless manual browsing, and traditional keyword‑based crawlers fail because sellers overload keywords, hide defects in images, and use deceptive pricing.

Why AI Is Needed for Xianyu Scraping

Keyword traps : searches for "iPhone" return unrelated accessories.

Image nuance : condition details appear only in photos.

Price deception : listed price may differ drastically from actual cost.

The ai-goofish-monitor project tackles these unstructured challenges by feeding item data to a large language model (LLM) for semantic understanding.

How It Works

1. Simulated Browsing with Playwright

Playwright, a Microsoft‑maintained browser automation tool, drives a real Chrome instance to log in, search, and paginate through listings. Users export their session cookies so the script can reuse an authenticated browser, avoiding API blocks.

2. Dual Visual‑Text Analysis via AI

For each captured product, the title, description, and images are packaged and sent to an LLM (OpenAI, Claude, Gemini, DeepSeek, etc.). Users can issue natural‑language prompts such as:

"I need iPhone 13 in condition 9+ with no screen scratches, battery health ≥ 90%, exclude sellers and resellers, only personal listings."

The model reviews the details, interprets image cues, and returns judgments like "seller disguise", "screen crack", or "worth buying".

3. Visualization and Automation

A web UI, also AI‑assisted, lets users:

Create tasks in natural language without writing regex.

Receive AI‑generated scores with explanations.

Schedule runs via Cron expressions (e.g., every 10 minutes).

"Vibe Coding": AI‑Generated Code

The author notes that over 90 % of the repository’s source files were produced by AI, including pull‑request changes. This exemplifies the emerging "Vibe Coding" trend, but also raises a "self‑validation dilemma": reviewing AI‑written code with another AI can create a black‑box without human scrutiny.

Objective Evaluation: Tool or Toy?

Advantages :

High precision: LLM filtering out junk outperforms simple keyword filters.

Low entry barrier: Docker one‑click deployment and web UI enable non‑programmers to use it (assuming they can obtain cookies).

Strong extensibility: supports multiple models and can integrate local LLMs like Ollama to cut API costs.

Limitations and Risks :

Platform anti‑bot measures may trigger captchas or rate limits despite Playwright’s human‑like behavior.

API cost: using GPT‑4 or Claude 3.5 for every item can outweigh the savings from a successful purchase; cheaper models such as DeepSeek are recommended.

Environment dependency: Docker setup and manual cookie extraction pose a technical hurdle for absolute beginners.

Users are reminded to respect Xianyu’s terms of service, avoid high‑frequency scraping, and never employ the tool for illegal activities.

Reference : GitHub project https://github.com/Usagi-org/ai-goofish-monitor