Tagged articles
Scraping
13 articles · Page 1 of 1
DrissionPage is a Python web‑automation library that merges browser control with high‑speed request handling, offering features like driver‑free operation, iframe support, multi‑tab management, full‑page screenshots, and an easy‑to‑use API demonstrated with a concise code example for web scraping tasks.
Scrapingbrowser automationdrissionpage
0 likes · 4 min read

DrissionPage is a Python-based web automation library that merges browser-driven interaction with high‑efficiency request handling, offering a fast, driver‑free, and feature‑rich alternative to Selenium for tasks such as scraping, element searching, and full‑page screenshots.
Scrapingdrissionpageselenium-alternative
0 likes · 5 min read

DrissionPage is a Python-based web automation library that merges browser-driven interaction with high‑efficiency request handling, offering a concise syntax, faster execution, and features like iframe handling, shadow‑root processing, and full‑page screenshots, making web scraping and testing more accessible.
Scrapingdrissionpageselenium-alternative
0 likes · 5 min read

This article explains how to create a Python‑based auto‑like robot for web sites by simulating login, handling cookies, and sending POST requests to platform APIs, while also covering common challenges like captchas and IP anti‑scraping measures.
CookieLike BotPython
0 likes · 7 min read

This tutorial demonstrates how to use Python's requests library to programmatically log into Weibo, publish a status update, and automatically like posts by handling cookies, sessions, and the required st token, complete with full code examples.
APIPythonScraping
0 likes · 10 min read

This article presents a complete Python web crawler example that extracts Baidu Baike entries, detailing the implementation of URL management, page downloading, HTML parsing with BeautifulSoup, data collection, and output generation, along with sample code and usage instructions.
BaikeBeautifulSoupScraping
0 likes · 9 min read

The article outlines several anti‑crawling techniques—including IP restrictions, User‑Agent validation, CAPTCHAs, AJAX loading, noscript tags, and cookie checks—while also offering practical advice for writing ethical, efficient, and robust web crawlers.
CAPTCHAIP blockingScraping
0 likes · 6 min read

This article demonstrates a step‑by‑step Python Selenium solution for completely automating Taobao login, covering dependency setup, class design, handling the sliding captcha, and alternative login via Sina Weibo, complete with full source code and usage instructions.
PythonScrapingTaobao
0 likes · 14 min read

This article explains how to update and use a Java HttpClient‑based spider that removes duplicate links, handles legacy page formats, extracts movie metadata and download URLs (magnet, ed2k, Baidu Pan), and stores the results in a MySQL database, with complete source code examples.
HttpClientJavaMySQL
0 likes · 12 min read

This article explains how to design a modular Python web crawler by breaking the system into five core components—scheduler, URL manager, downloader, parser, and data storage—provides detailed code examples for each module, and demonstrates a complete end‑to‑end crawling workflow on a sample website.
PythonScrapingbackend development
0 likes · 12 min read

This article walks through building a high‑performance novel‑site crawler using MongoDB, detailing the extraction of category links, multi‑process/thread URL management, database operations, and the final deduplication results, while illustrating each step with diagrams and screenshots.
MongoDBMultithreadingScraping
0 likes · 4 min read


21CTONov 20, 2016 · Backend Development
This article explores the fundamentals and advanced techniques of building web crawlers, covering crawler types, essential features, RSS/ATOM harvesting, custom scraping methods, PHP header manipulation, regex extraction, and concurrency, providing actionable code examples for backend developers.
RSSScrapingWeb Crawling
0 likes · 9 min read


ITPUBMay 6, 2016 · Backend Development
This guide compares Scrapy (especially version 0.16) with gevent‑based crawling solutions, outlines their strengths, weaknesses, and common pitfalls, and provides practical tips, resource links, and deployment advice for building efficient Python web scrapers.
BackendGeventPython
0 likes · 11 min read
