This article presents a concise guide to Python web scraping, covering eight practical techniques—including basic GET/POST requests, proxy usage, cookie management, header spoofing, page parsing, captcha handling, gzip compression, and multithreaded crawling—each illustrated with clear code examples.
This article presents a comprehensive guide to Python web scraping, covering basic GET and POST requests with urllib2, using proxy IPs, managing cookies, disguising as a browser via custom headers, handling gzip-compressed responses, and accelerating crawls with a simple multithreaded worker pool.
This tutorial shows how to capture the network packets of the Super Course Table Android app, log in by sending the required headers and encrypted credentials, and continuously fetch user‑generated topics using Python's urllib2 and JSON parsing, enabling infinite scrolling data extraction.
This tutorial demonstrates how to capture network packets from the Super Schedule Android app, perform a login request with required headers, and repeatedly fetch and parse topic data using Python urllib2 and JSON handling, enabling infinite scrolling of user‑generated topics.
This tutorial demonstrates how to programmatically log into a website by handling cookies, fetching and solving captchas, and posting the required form data using Python's urllib2 and cookielib modules, with a complete example targeting a school academic system.
This article provides a comprehensive Python web‑scraping guide covering basic GET/POST requests with urllib2, proxy handling, cookie management, header manipulation to mimic browsers, gzip compression handling, regular‑expression and library parsing, simple captcha strategies, and a multithreaded thread‑pool example.
This tutorial explains how to analyze the Ant Short‑Term Rental website's anti‑scraping mechanisms, extract the required Cookie and User‑Agent headers, and use Python's urllib2 and BeautifulSoup to reliably crawl rental listings, save the data to CSV, and optionally extend the scraper with Selenium.
This article provides a comprehensive guide to Python web scraping, covering basic GET/POST requests, proxy usage, cookie management, browser header spoofing, gzip compression handling, and multithreaded crawling to improve efficiency and avoid common obstacles.
This article explains how to use Python's urllib2 library for web scraping, covering basic page fetching, proxy configuration, cookie handling, form submission, header manipulation, anti‑hotlink techniques, and advanced methods like Selenium to bypass restrictions.
This tutorial shows how to fetch a joke via a free API using Python's urllib2, split the short joke to fit a 140‑character SMS limit, send it through a QQ email to a 139 mailbox, and schedule the script with a Linux crontab entry for daily delivery.
This guide walks through eight practical Python urllib2 techniques for web crawling, covering basic GET/POST requests, proxy usage, cookie management, header spoofing, page parsing with regex and BeautifulSoup, captcha handling, gzip compression, and multithreaded fetching with a simple thread pool.
This article provides a comprehensive Python web scraping guide covering basic page retrieval with GET and POST, proxy usage, cookie handling, header spoofing, page parsing techniques, captcha processing, gzip compression, and multithreaded crawling, complete with code snippets for each step.
This guide walks through practical Python web‑scraping techniques using urllib2, covering basic GET/POST requests, proxy handling, cookie management, header spoofing, page parsing with regex/lxml/BeautifulSoup, simple captcha solutions, gzip compression, and a multithreaded crawling template.
This guide walks through using Python's urllib2 library for web crawling, covering basic GET/POST requests, handling proxy IPs, managing cookies, spoofing browser headers, processing gzip-compressed responses, and implementing multithreaded fetching with a simple thread‑pool template.
This tutorial shows iOS developers how to set up Python on macOS, install pip and BeautifulSoup, fetch a web page with urllib2, parse the HTML to locate image tags, extract their URLs, and download the images programmatically for use in an app.
This guide dives deep into urllib2, covering proxy configuration, timeout handling, custom headers, redirects, cookies, PUT/DELETE methods, response codes, debugging logs, form processing, browser impersonation, and anti-hotlinking tricks, providing practical code snippets for each feature.
This guide explains urllib2's info() and geturl() methods, demonstrates how to work with Openers and Handlers, and shows how to implement HTTP Basic Authentication with practical Python examples.
This article explains how to use Python's urllib2 module to fetch web pages, covering basic request handling, creating Request objects, sending GET and POST data, setting custom headers, and demonstrates practical examples with code snippets and screenshots.
This guide walks through practical Python web‑scraping techniques using urllib2, covering basic page fetching, proxy usage, cookie handling for logins, form submission, header spoofing, anti‑hotlink tricks, multithreaded crawling, and strategies for bypassing simple captchas, all illustrated with code snippets.
