How to Use Python to Scrape and Export Your Weibo Favorites

Python Implementation

Goal: download Sina Weibo favorites (posts, comments, images, video links) and convert them into a document.

Approach:

Simulate login to Weibo and save cookies.

Use the cookies to request the favorites page URL.

Iterate from the first page to the last, downloading each page.

Two processing steps are performed:

A. Directly download the HTML page (saved locally; viewing requires network for JS, images, etc.).

B. Parse the required Weibo content; currently the raw data is stored for later processing.

Future work will use lxml with XPath to extract text, images, and video links and convert them into a document format.

Login script reference: http://www.douban.com/note/201767245/ (used directly).

Steps

Open your Weibo favorites page (e.g., http://weibo.com/fav?leftnav=1&wvr=3.6&page=1) and note the URL prefix.

Modify the script with your username, password, and the URL prefix.

Run the script: python weibo_collect.py Result screenshots:

The variable tmpcontent contains all extracted Weibo information, awaiting further formatting.