Learn how to overcome anti‑scraping defenses by using Python's requests_html library to fetch and parse dynamic web pages, with a complete code example that extracts company names from a target site, plus tips on handling cookies, headers, and Unicode decoding.
This article walks through a step‑by‑step Python solution for scraping Shanghai Gold Exchange transaction data, using requests_html to collect URLs, pandas to parse HTML tables, and finally exporting the aggregated results to Excel, while also offering practical tips for sharing code and debugging.
This article demonstrates two Python approaches—using high-level requests and low-level HTTPConnection—to fetch intraday stock data from Eastmoney, providing complete code examples, URL discovery steps, and practical tips for data‑mining enthusiasts.
This article walks through a real‑world Python web‑scraping case, explaining the problem, revealing hidden request URLs, showing the complete code, and demonstrating how to retrieve and verify data from Baidu's Aiqicha platform.
This tutorial explains how to use Python's requests and lxml libraries to crawl a website, bypass SSL verification, extract audio file URLs, and download sound effect MP3s in bulk, providing complete code snippets and practical tips for safe web scraping.
This tutorial walks you through creating a Python-powered QQ Music scraper that retrieves song details, lyrics, and comments, generates a word‑cloud visualization, and packages the functionality into a menu‑driven command‑line tool, complete with code snippets and troubleshooting tips.
This article walks through using Python to download all WeChat group member avatars, save them locally, and then apply Baidu's face‑recognition API to evaluate age, gender and beauty scores, finally filtering and ranking the images based on the results.
This tutorial shows how to log in to Douban with Python, scrape short comments for the animated film “Nezha”, process the text using jieba for Chinese word segmentation, and generate a visual word cloud that highlights the most frequent audience sentiments.
This article walks through the complete Scrapy implementation for extracting WeChat Moments data, covering item definition, spider configuration, request handling, parsing logic, pipeline setup, execution commands, and encoding fixes to produce a clean JSON output.
