How to Crawl Baidu Tieba Images with Python and Save Them Locally

Overview

This tutorial demonstrates how to use Python to retrieve Baidu Tieba pages, extract image URLs with regular expressions, and save the images to the local file system.

1. Retrieve the whole page

The urllib module provides functions to read web page data similar to reading local files. A getHtml(url) function opens the URL with urllib.urlopen(), reads the content with read(), and returns the full HTML.

2. Filter the desired data

Python’s powerful re module is used to locate image links. First, compile a regular expression with re.compile(), then apply re.findall() to the HTML returned by getHtml(). The getImg(html) function extracts all image URLs that match the pattern, such as src="http://imgsrc.baidu.com/forum…jpg".

3. Save images locally

Iterate over the list of image URLs with a for loop, download each image, and rename it using a sequential counter (e.g., x+1) to keep filenames orderly. Images are saved in the program’s directory by default. After the script finishes, the downloaded files appear in the local folder.