How to Crawl Baidu Tieba Images with Python and Save Locally

Python script to download Baidu Tieba images

Description: Use Python to crawl Baidu Tieba images and save them locally. The author is a beginner in web crawling and finds regular expressions challenging.

1. Retrieve the entire page data

The urllib module provides an interface to read web page data. A getHtml() function is defined that uses urllib.urlopen() to open a URL and read() to download the page content, which is then printed.

2. Filter the desired data from the page

Python's powerful regular expression engine is used. After locating image URLs in the page source (e.g., src="http://imgsrc.baidu.com/forum…jpg"), a getImg() function extracts the image links. The re module functions re.compile() and re.findall() compile the pattern and find all matches.

3. Save the filtered data locally

A for loop iterates over the extracted image URLs, renames each file using an incrementing variable to keep filenames consistent, and saves them to the program's directory. After the script finishes, the downloaded images appear in the local folder.

Author: MTbaby Source: CSDN Blog