How to Scrape and Export Your Sina Weibo Favorites with Python

Using Python to Export Sina Weibo Favorites

The author discovered over 2,000 collected Weibo items spanning 104 pages and wanted to automate their extraction into a document.

Goal

Weibo post text and comments

Images

Video links

Approach

1. Simulate login to Sina Weibo and save the authentication cookie.

2. Use the cookie to request the favorites page URLs.

3. Iterate from the first page to the last, performing two actions for each page:

Download the raw HTML (requires internet to view dynamic content).

Parse and store the needed Weibo content for later processing.

Future processing will use lxml with XPath to convert the saved HTML into a document, including images and video links.

Implementation Details

The login simulation follows the method described at http://www.douban.com/note/201767245/, which the author copied directly.

Steps

Open your Weibo account, go to the “Favorites” section (e.g., http://weibo.com/fav?leftnav=1&wvr=3.6&page=1) and note the URL prefix.

Edit the script with your username, password, and the URL prefix.

Run the script: python weibo_collect.py The script produces a tmpcontent file containing all raw Weibo information, which can later be parsed into the desired format.

Sample output screenshots are shown below: