Scrape and Filter Zhihu “Beautiful Women” Images with Python & Baidu AI

1. Data source

All images appearing in answers to questions under Zhihu topic “美女”.

2. Crawling tool

Python 3 with third‑party libraries Requests, lxml, and Baidu AipFace; the script is about 100 lines.

3. Required environment

Mac / Linux / Windows (Windows may need filename character filtering).

No Zhihu login required.

Baidu Cloud account for the face detection service.

4. Face detection library

AipFace, a Python SDK from Baidu AI Open Platform, provides HTTP‑based face detection and is free to use.

5. Filtering criteria

Discard images without faces (e.g., landscapes, non‑portrait photos).

Keep only female faces; male images are mostly celebrities and are ignored.

Exclude non‑real persons such as anime characters (AipFace human confidence < 0.6).

Remove low‑beauty scores (AipFace beauty < 45) to save storage.

6. Implementation logic

Use Requests to fetch a list of discussions under the “美女” topic.

Parse each discussion’s HTML with lxml to extract all img tag src attributes.

Download each image via Requests (ignoring animated GIFs).

Send the image to AipFace for face detection.

Apply the filtering rules from section 5.

Save the remaining images to the local file system with filenames composed of beauty score, author, question title, and an index.

Repeat from step 1 until the desired amount is collected.

7. Crawl results

Images are stored in a folder; the highest beauty score observed (aside from “angelababy”) is 88, though the ranking is subjective.

9. Preparation to run

Install Python 3.

Install requests, lxml, baidu-aip via pip (single command).

Apply for a free Baidu Cloud face detection service.