BestHub
Sign in
Tagged articles

Web Scraping

484 articles · Page 1 of 5
Wuming AI
Wuming AI
Aug 9, 2026 · Artificial Intelligence

10 Essential AI Skills I Use Daily (Don’t Hoard Unneeded Skills)

The author curates ten practical AI Skills—ranging from web capture and article deep‑reading to local audio transcription, skill optimization, engineering method libraries, WeChat formatting, video creation, batch photo downloading, and skill discovery—explaining their problem scope, key features, usage steps, and real‑world observations such as token savings, star counts, and limitations.

AI SkillsAudio TranscriptionAutomation
0 likes · 12 min read
10 Essential AI Skills I Use Daily (Don’t Hoard Unneeded Skills)
Node.js Tech Stack
Node.js Tech Stack
Aug 8, 2026 · Cloud Native

Why Chrome Is Too Heavy for Agents and How Cloudflare Built a Rust + V8 Agent Browser

The article examines Cloudflare's Kitesurf, a lightweight Rust‑and‑V8 based browser engineered for server‑side agents, detailing its stripped‑down architecture, performance trade‑offs versus Chromium, rapid 12‑week development aided by AI, and its implications for cost‑effective automated web tasks.

Agent BrowserCloudflare WorkersKitesurf
0 likes · 11 min read
Why Chrome Is Too Heavy for Agents and How Cloudflare Built a Rust + V8 Agent Browser
IoT Full-Stack Technology
IoT Full-Stack Technology
Jul 30, 2026 · Backend Development

Build a Super‑Simple Node.js Web Scraper for Douban Top 250

This tutorial walks through creating a minimal Node.js crawler that fetches the Douban Top 250 movie page using the built‑in https module, parses the HTML with cheerio, extracts titles, ratings and images, stores the data in an array, and writes the results to a JSON file.

HTTPSTutorialWeb Scraping
0 likes · 5 min read
Build a Super‑Simple Node.js Web Scraper for Douban Top 250
Python Crawling & Data Mining
Python Crawling & Data Mining
Jul 25, 2026 · Backend Development

Comparing Python’s Four Web‑Scraping Selectors: Regex, BeautifulSoup, XPath, and CSS

This article reviews the four main Python selectors used for web scraping—regular expressions, BeautifulSoup, lxml/XPath, and CSS—detailing their installation, parsing behavior, advantages, drawbacks, and a performance comparison table to help developers choose the most suitable tool for their projects.

BeautifulSoupCSS selectorsPython
0 likes · 8 min read
Comparing Python’s Four Web‑Scraping Selectors: Regex, BeautifulSoup, XPath, and CSS
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 29, 2026 · Backend Development

How the New z‑skills Video Download Split Improves Knowledge‑Gathering Workflows

The author refactors the z‑skills suite by separating video link discovery (z‑web‑pack) from actual downloading (z‑video‑downloader), explains why the split solves performance and reliability issues, details four concrete code and documentation changes, and shows a simple usage pattern for the updated skills.

PythonWeb Scrapingtool architecture
0 likes · 7 min read
How the New z‑skills Video Download Split Improves Knowledge‑Gathering Workflows
AI Architecture Path
AI Architecture Path
Jun 27, 2026 · Backend Development

Turn Your AI Agent into a Web‑Data Pro with Firecrawl’s 139K‑Star Open‑Source Scraper

Firecrawl is a 139K‑star open‑source web‑scraping API that handles dynamic JavaScript pages, full‑site crawling, search, and interactive browsing, offers built‑in proxy rotation and LLM‑ready Markdown/JSON output, and provides detailed code samples and deployment guides that outperform traditional tools like Scrapy and Selenium.

AI agentsFirecrawlLLM Integration
0 likes · 12 min read
Turn Your AI Agent into a Web‑Data Pro with Firecrawl’s 139K‑Star Open‑Source Scraper
AI Engineering
AI Engineering
Jun 23, 2026 · Industry Insights

After Three Years of Building Scrapers, This One‑Line Tool Beats It

The article analyzes why acquiring public social‑media data is the biggest obstacle for AI‑driven content apps, compares DIY crawlers with commercial solutions, and demonstrates how RedFox’s unified API lets developers fetch multi‑platform data with a single key, minimal code, and lower maintenance costs.

AI dataPythonREST
0 likes · 10 min read
After Three Years of Building Scrapers, This One‑Line Tool Beats It
Ubiquitous Tech
Ubiquitous Tech
Jun 13, 2026 · Artificial Intelligence

How a Programmer Built an AI Agent Skill to Watch the 2026 World Cup

A developer created an open‑source AI Agent Skill that scrapes real‑time data from the 2026 North‑American World Cup, normalizes it into JSON, and lets anyone ask natural‑language questions about matches, teams, players, and statistics, all with a single command‑line install.

AI AgentNode.jsOpen Source
0 likes · 9 min read
How a Programmer Built an AI Agent Skill to Watch the 2026 World Cup
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 6, 2026 · Artificial Intelligence

How to Build a Personal Knowledge Base with My Custom web‑pack Skill

This article explains how to construct a personal knowledge base using the author’s open‑source web‑pack Skill, which automates raw material collection, image localization, link expansion, and structured output, addressing the limitations of Obsidian’s Web Clipper and aligning with Karpathy’s LLM Wiki three‑layer architecture.

AI agentsAutomationKnowledge Management
0 likes · 9 min read
How to Build a Personal Knowledge Base with My Custom web‑pack Skill
liandk
liandk
May 17, 2026 · Fundamentals

Advanced Python Regex & Error Debugging: Extract Complex Data in 5 Minutes

This article extends basic regex knowledge with three advanced techniques, demonstrates how to troubleshoot module‑mirror, URL‑format, and content‑limit errors across nine sample documents, and provides ready‑to‑copy Python code for precise data extraction while avoiding common pitfalls.

PythonRegexURL validation
0 likes · 14 min read
Advanced Python Regex & Error Debugging: Extract Complex Data in 5 Minutes
liandk
liandk
May 13, 2026 · Backend Development

Python Web Scraping Basics: Extract Strings and Intro to Regex in 5 Minutes

This tutorial walks through fixing common requests‑module errors, selecting a reliable PyPI mirror, formatting URLs correctly, and using built‑in string methods and basic regex to reliably extract IP, keyword and request URL data from HTTP responses, with batch examples and a summary of four common pitfalls.

PythonRegexString Extraction
0 likes · 12 min read
Python Web Scraping Basics: Extract Strings and Intro to Regex in 5 Minutes
liandk
liandk
May 13, 2026 · Backend Development

Advanced Requests and Simple Web Scraping in Python: Extract Data in 5 Minutes

This tutorial walks you through installing the requests library from reliable mirrors, setting browser-like headers, extracting JSON or raw text data, handling common errors, and performing batch requests to build a quick, functional web scraper in just five minutes.

PythonWeb Scrapingbatch-requests
0 likes · 11 min read
Advanced Requests and Simple Web Scraping in Python: Extract Data in 5 Minutes
liandk
liandk
May 13, 2026 · Fundamentals

Python Basics – Part 8: Master requests in 5 Minutes to Start Web Scraping

This tutorial walks you through installing the Python requests library, demonstrates basic GET requests, adding query parameters, handling errors with timeout and exceptions, and highlights four common pitfalls with solutions, enabling you to quickly build simple web‑scraping scripts in just five minutes.

HTTPPythonWeb Scraping
0 likes · 8 min read
Python Basics – Part 8: Master requests in 5 Minutes to Start Web Scraping
liandk
liandk
May 10, 2026 · Fundamentals

Python Basics in 3 Minutes: What It Can Do and How to Get Started

This guide explains Python's main uses—automation, web scraping, and simple web apps—walks through Windows/macOS installation, shows how to run the first program, lists four common beginner pitfalls with solutions, and summarizes key takeaways for rapid onboarding.

AutomationBeginner TutorialCommon pitfalls
0 likes · 4 min read
Python Basics in 3 Minutes: What It Can Do and How to Get Started
AI Architecture Path
AI Architecture Path
May 6, 2026 · Backend Development

Scrapling: Self‑Healing Web Scraper That Bypasses Cloudflare and Is 784× Faster Than BS4

Scrapling is an open‑source, adaptive web‑scraping framework that automatically tracks element changes, bypasses Cloudflare and other anti‑scraping defenses, offers multiple fetchers (including stealth mode), and delivers extraction speeds up to 784× faster than BeautifulSoup (BS4) while supporting concurrency, AI integration, and easy CLI usage.

PythonScraplingWeb Scraping
0 likes · 16 min read
Scrapling: Self‑Healing Web Scraper That Bypasses Cloudflare and Is 784× Faster Than BS4
AI Engineering
AI Engineering
May 3, 2026 · Backend Development

Rust‑Based Headless Browser Uses 85% Less Resources Than Chrome

Obscura, an open‑source Rust headless browser designed for AI agents and large‑scale crawling, cuts memory usage by 85%, reduces binary size, speeds page loads to 85 ms, starts instantly, includes strong anti‑detection features, and works with the Chrome DevTools Protocol as a drop‑in replacement for Headless Chrome.

CDPObscuraRust
0 likes · 5 min read
Rust‑Based Headless Browser Uses 85% Less Resources Than Chrome
Wu Shixiong's Large Model Academy
Wu Shixiong's Large Model Academy
Apr 14, 2026 · Artificial Intelligence

Designing High‑Quality Tools for Deep Research Agents: From Search to Python Execution

This article explains how to turn simple API calls into robust, noise‑filtering tools—Search, Visit, Scholar, and Python—by adding domain blacklists, relevance scoring, query‑driven extraction, safety sandboxes, and a unified registry, ultimately boosting the success rate of LLM‑driven research agents.

AI agentsLLM safetyReAct
0 likes · 32 min read
Designing High‑Quality Tools for Deep Research Agents: From Search to Python Execution
AI Architecture Path
AI Architecture Path
Mar 22, 2026 · Backend Development

How Lightpanda Delivers 11× Faster Headless Browsing with 1/9 Memory Usage

Lightpanda, an open‑source headless browser written in Zig, claims up to eleven times faster execution and one‑ninth the memory of Chrome, offering a lightweight architecture for large‑scale web scraping, AI agent testing, and automation with detailed benchmarks, feature lists, usage commands, and known limitations.

Web ScrapingZigheadless browser
0 likes · 9 min read
How Lightpanda Delivers 11× Faster Headless Browsing with 1/9 Memory Usage
Code Wrench
Code Wrench
Mar 16, 2026 · Backend Development

Mastering Chromedp: Go‑Based Chrome Automation for Real‑World Use Cases

This article provides a comprehensive guide to using the Go library chromedp for Chrome DevTools Protocol‑driven browser automation, covering its core architecture, practical techniques for navigation, network interception, anti‑bot evasion, performance tuning, and production‑grade recommendations with full code examples.

Chrome DevTools ProtocolGoWeb Scraping
0 likes · 10 min read
Mastering Chromedp: Go‑Based Chrome Automation for Real‑World Use Cases
Su San Talks Tech
Su San Talks Tech
Mar 2, 2026 · Backend Development

How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping

This guide explains how OpenClaw’s built‑in browser feature and the optional Browser Relay extension enable effortless data extraction from both public and login‑required websites, providing step‑by‑step installation commands, Chrome extension loading instructions, and a practical example of scraping top posts from a social platform.

Chrome ExtensionOpenClawWeb Scraping
0 likes · 6 min read
How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping
Old Meng AI Explorer
Old Meng AI Explorer
Jan 22, 2026 · Industry Insights

Three Must‑Try Open‑Source AI Tools for Scraping, PPT Creation, and Video Generation

This article reviews three recently popular open‑source AI tools—Spider_XHS for Xiaohongshu data scraping, PPTAgent for one‑click presentation generation, and Code2Video for code‑driven video creation—detailing their key features, quick‑start instructions, and GitHub repositories, all free and ready to deploy.

AI ToolsAutomationOpen Source
0 likes · 7 min read
Three Must‑Try Open‑Source AI Tools for Scraping, PPT Creation, and Video Generation
AI Insight Log
AI Insight Log
Jan 4, 2026 · Artificial Intelligence

How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt

The article examines the open‑source ai‑goofish‑monitor project, which combines Playwright‑driven browsing with large‑language‑model analysis to continuously scan Xianyu listings, filter out junk, and highlight high‑quality items, while also discussing its AI‑generated code, benefits, limitations, and security risks.

AIAutomationLLM
0 likes · 7 min read
How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt
Open Source Tech Hub
Open Source Tech Hub
Dec 28, 2025 · Backend Development

Automate Chrome with PHP: Install and Use ChromePHP for Screenshots, PDFs, and More

ChromePHP is a PHP library that integrates the Chrome DevTools protocol, allowing developers to control Chrome or Chromium directly from PHP code for tasks such as navigation, JavaScript execution, screenshots, PDF generation, and input simulation, with simple Composer installation and flexible configuration.

Chrome automationChromePHPDevTools
0 likes · 3 min read
Automate Chrome with PHP: Install and Use ChromePHP for Screenshots, PDFs, and More
Old Meng AI Explorer
Old Meng AI Explorer
Dec 10, 2025 · Operations

How Spider_XHS Turns Xiaohongshu Data Collection into a 10× Efficiency Boost

Spider_XHS is an open‑source Xiaohongshu crawler that automates note, user, comment, and message extraction, offers watermark‑free media downloads, exports structured Excel/JSON data, integrates with the creator platform, and includes proxy and anti‑ban features, enabling marketers and researchers to cut weeks of manual work into hours.

AutomationData CollectionOpen Source
0 likes · 10 min read
How Spider_XHS Turns Xiaohongshu Data Collection into a 10× Efficiency Boost
Python Programming Learning Circle
Python Programming Learning Circle
Oct 16, 2025 · Backend Development

Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation

This article introduces DrissionPage, a Python library that merges Selenium and Requests, explains its three page objects, highlights seamless mode switching, built‑in utilities, API changes in version 4.0, and provides practical code examples for web automation, data crawling, and testing.

Web AutomationWeb Scrapingbrowser automation
0 likes · 14 min read
Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 5, 2025 · Artificial Intelligence

How Browser-Use Leverages LLMs to Transform Browser Automation

This article explores Browser-Use, an AI‑driven browser automation framework that combines large language models, visual perception, and DOM analysis to enable intelligent, multi‑step web tasks such as registration, price comparison, form filling, and monitoring, while detailing its architecture, historical context, core modules, and future challenges.

AI agentsLLMLangChain
0 likes · 26 min read
How Browser-Use Leverages LLMs to Transform Browser Automation
37 Interactive Technology Team
37 Interactive Technology Team
Sep 3, 2025 · Artificial Intelligence

How AI is Revolutionizing Web Scraping: Tools, Techniques, and Best Practices

Discover how AI, especially large language models, transforms traditional web scraping by introducing semantic understanding, dynamic adaptability, and automated extraction, with in‑depth reviews of emerging tools like Crawl4AI and Browser‑use, practical code examples, best‑practice guidelines, and deployment tips for modern data collection.

AIAutomationBrowser Use
0 likes · 17 min read
How AI is Revolutionizing Web Scraping: Tools, Techniques, and Best Practices
21CTO
21CTO
Aug 19, 2025 · Artificial Intelligence

How Crawl4AI Transforms Web Scraping with AI‑Powered Automation

Crawl4AI is an open‑source AI agent that automates web crawling and data extraction, offering free usage, intelligent parsing, structured JSON/Markdown output, and versatile features like scrolling, multi‑URL scraping, media and metadata extraction, all demonstrated through step‑by‑step Python examples and integration with AI agents.

AI agentsAutomationCrawl4AI
0 likes · 8 min read
How Crawl4AI Transforms Web Scraping with AI‑Powered Automation
Python Programming Learning Circle
Python Programming Learning Circle
Jul 24, 2025 · Backend Development

Boost Your Python Productivity: 8 Must‑Use Libraries for Faster Development

Discover eight essential Python libraries—Rich, Typer, Pendulum, Pydantic, Faker, tqdm, Requests‑HTML, and Loguru—that transform logging, CLI creation, date handling, data validation, mock data generation, progress tracking, web scraping, and logging, showing how leveraging existing tools can dramatically increase development efficiency.

LibrariesPythonWeb Scraping
0 likes · 8 min read
Boost Your Python Productivity: 8 Must‑Use Libraries for Faster Development
DevOps
DevOps
Jun 16, 2025 · Artificial Intelligence

Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code

This article reviews six open‑source MCP servers—Bright Data, Graphiti, GitIngest, Terminal, Code Executor, and MindsDB—showing how each extends large language models with web scraping, long‑term memory, code navigation, command‑line control, sandboxed Python execution, and multi‑source data integration, complete with practical code examples.

AI ToolsKnowledge GraphMCP
0 likes · 9 min read
Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code
DataFunTalk
DataFunTalk
Jun 11, 2025 · Backend Development

Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration

This article provides a comprehensive overview of web‑scraping technologies, compares popular tools such as requests, BeautifulSoup and Selenium, introduces AI‑assisted crawling with DeepSeek, and walks through practical steps for using BrightData’s platform to collect industry data, complete with ready‑to‑run Python code.

BeautifulSoupBrightDataDeepSeek
0 likes · 13 min read
Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration
Python Programming Learning Circle
Python Programming Learning Circle
Jun 7, 2025 · Backend Development

Master Python Web Scraping: From Requests to Selenium and Scrapy

Learn how to efficiently scrape web pages using Python by exploring multiple approaches—including simple requests with BeautifulSoup, fast parsing with lxml, dynamic content extraction with Selenium, and large‑scale crawling with Scrapy—complete with installation steps, code snippets, and detailed explanations.

BeautifulSoupPythonScrapy
0 likes · 10 min read
Master Python Web Scraping: From Requests to Selenium and Scrapy
Python Programming Learning Circle
Python Programming Learning Circle
May 29, 2025 · Big Data

Common Python Web Scraping Techniques for E‑commerce Data Collection

This article introduces ten practical Python-based web scraping methods—including requests, Selenium, Scrapy, Crawley, PySpider, aiohttp, asks, vibora, Pyppeteer, and Fiddler‑plus‑Node reverse engineering—explaining their use cases, advantages, and code examples for efficiently gathering e‑commerce and app data.

Data CollectionScrapyWeb Scraping
0 likes · 8 min read
Common Python Web Scraping Techniques for E‑commerce Data Collection
php Courses
php Courses
May 14, 2025 · Backend Development

Python Advantages for Web Scraping and Core Library Guide

This article outlines Python's advantages for web crawling, introduces core libraries such as Requests, BeautifulSoup, and Scrapy, details a step-by-step development workflow, provides practical code examples for extracting news titles, and highlights important considerations and advanced techniques for robust scraper implementation.

BeautifulSoupPythonScrapy
0 likes · 5 min read
Python Advantages for Web Scraping and Core Library Guide
Python Programming Learning Circle
Python Programming Learning Circle
Apr 30, 2025 · Backend Development

Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib

This article demonstrates how to use Python's requests and BeautifulSoup libraries to scrape current and 14‑day weather data from China Weather, store the results in CSV files, and perform comprehensive visual analysis—including temperature, humidity, AQI, wind direction, and forecast charts—using pandas, numpy, and matplotlib.

Data VisualizationWeather AnalysisWeb Scraping
0 likes · 26 min read
Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib
Subtle Storm
Subtle Storm
Apr 17, 2025 · Operations

Bypass Slider Captcha in Python Selenium Login Automation

This guide shows how to use Python Selenium to automate login with slider captcha protection by installing dependencies, initializing ChromeDriver, locating elements, generating a human‑like drag trajectory, performing the drag action, and verifying successful login, including advanced image‑processing tips.

AutomationPythonWeb Scraping
0 likes · 8 min read
Bypass Slider Captcha in Python Selenium Login Automation
Python Programming Learning Circle
Python Programming Learning Circle
Apr 1, 2025 · Backend Development

Implementing Local QR Code Login for QQ Music Using Python

This tutorial walks through building a Python script that fetches a QQ Music QR code, displays it locally, monitors its status, extracts required encrypted parameters from cookies, and completes the login process by repeatedly polling the authentication endpoint to obtain a session.

AutomationQR loginWeb Scraping
0 likes · 13 min read
Implementing Local QR Code Login for QQ Music Using Python
DataFunSummit
DataFunSummit
Feb 25, 2025 · Artificial Intelligence

Collecting High-Quality LLM Training Data and Custom Model Training Guide

This article explains what constitutes high‑quality LLM training data, why large datasets are essential, outlines the step‑by‑step process for collecting, preprocessing, and fine‑tuning models, and highlights the best data sources—including web content, books, code repositories, and news—while noting available free datasets.

AIData CollectionLLM
0 likes · 9 min read
Collecting High-Quality LLM Training Data and Custom Model Training Guide
DataFunSummit
DataFunSummit
Feb 13, 2025 · Big Data

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

This tutorial explains e‑commerce web scraping fundamentals, covering definitions, tool types, data categories, step‑by‑step Python script creation with Requests, BeautifulSoup, and Selenium, provides sample code for Amazon, Walmart, and eBay, discusses challenges like dynamic pages and anti‑scraping measures, and recommends using specialized scraping APIs.

BeautifulSoupBright DataPython
0 likes · 15 min read
E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges
php Courses
php Courses
Dec 26, 2024 · Backend Development

Python vs PHP for Web Scraping: A Comparative Guide

This article compares Python and PHP for web scraping, outlining each language's strengths, ecosystem, performance, learning curve, and community support to help readers decide which tool best fits their project requirements and experience level.

Web Scrapingdata extractionprogramming comparison
0 likes · 9 min read
Python vs PHP for Web Scraping: A Comparative Guide
Raymond Ops
Raymond Ops
Dec 23, 2024 · Backend Development

Master Python Proxy Techniques for Efficient Web Scraping

This guide explains the essential proxy concepts for Python web scraping, covering what proxies are, common types, how they protect crawlers, practical usage with the requests library, and the role of proxy pools in improving scraping efficiency.

Web Scrapingbackend development
0 likes · 7 min read
Master Python Proxy Techniques for Efficient Web Scraping
Python Programming Learning Circle
Python Programming Learning Circle
Nov 25, 2024 · Backend Development

Python Web Scraping of Fund Holdings Data and Analysis Using Requests, Selenium, and MongoDB

This tutorial demonstrates how to analyze a fund ranking website, construct dynamic URLs, extract six‑digit fund codes, crawl fund holding pages with requests and Selenium, store the results in MongoDB, and finally process the data to identify the most frequently held stocks across thousands of funds.

MongoDBWeb Scrapingselenium
0 likes · 13 min read
Python Web Scraping of Fund Holdings Data and Analysis Using Requests, Selenium, and MongoDB
Test Development Learning Exchange
Test Development Learning Exchange
Nov 23, 2024 · Operations

Comprehensive Python Automation Scripts for Common Tasks

This article presents a collection of practical Python scripts covering file management, web scraping, email sending, Excel handling, data cleaning, image processing, system monitoring, PDF manipulation, OCR, database interaction, social media posting, testing, and cloud storage, each with clear descriptions and ready‑to‑run code examples.

AutomationData ProcessingPython
0 likes · 12 min read
Comprehensive Python Automation Scripts for Common Tasks
Python Crawling & Data Mining
Python Crawling & Data Mining
Nov 8, 2024 · Fundamentals

Extracting Courier Addresses with Python: A Step‑by‑Step Guide

This article walks through a practical Python solution for parsing a list of courier records, extracting province information, grouping entries by location using dictionaries, and optionally applying regular expressions and pandas for further analysis, all illustrated with clear code examples and output screenshots.

DictionariesListsPandas
0 likes · 11 min read
Extracting Courier Addresses with Python: A Step‑by‑Step Guide
Python Programming Learning Circle
Python Programming Learning Circle
Nov 7, 2024 · Backend Development

11 Efficient Python Web Scraping Tools and a Practical News‑Site Example

This article introduces eleven powerful Python libraries for web scraping—including Requests, BeautifulSoup, Scrapy, Selenium, PyQuery, Lxml, Pandas, Pyppeteer, aiohttp, Faker, and ProxyPool—explains their key features, provides ready‑to‑run code snippets, and demonstrates a real‑world news‑site crawling case study.

BeautifulSoupWeb Scrapingrequests
0 likes · 13 min read
11 Efficient Python Web Scraping Tools and a Practical News‑Site Example
Python Crawling & Data Mining
Python Crawling & Data Mining
Nov 1, 2024 · Backend Development

Fixing Chinese Character Garbling in Python Web Scraping

This article walks through a real‑world Python web‑scraping issue where Chinese characters appear as garbled text, explains why the default encoding fails, and shows how setting the response’s apparent encoding resolves the problem, complete with sample code and practical tips for posting questions.

PythonWeb Scrapingencoding
0 likes · 5 min read
Fixing Chinese Character Garbling in Python Web Scraping
php Courses
php Courses
Oct 21, 2024 · Backend Development

Using PHP get_headers() to Retrieve HTTP Response Headers

This article explains the PHP get_headers() function, its syntax, parameters, and usage examples, demonstrates how to retrieve and display HTTP response headers from a URL, and discusses common application scenarios such as checking remote file existence, obtaining file metadata, and supporting web crawling or monitoring.

HTTPWeb Scrapingbackend
0 likes · 4 min read
Using PHP get_headers() to Retrieve HTTP Response Headers
Test Development Learning Exchange
Test Development Learning Exchange
Sep 28, 2024 · Fundamentals

Python Cheat Sheet for File Management, Data Processing, Web Requests, Automation, and Text Manipulation

This article provides a concise Python cheat sheet covering common file and directory operations, CSV/JSON data handling, web request and crawling techniques, automation tasks such as scheduling and email sending, and various string manipulation utilities, each illustrated with ready‑to‑run code snippets.

PythonScriptingWeb Scraping
0 likes · 10 min read
Python Cheat Sheet for File Management, Data Processing, Web Requests, Automation, and Text Manipulation
Python Programming Learning Circle
Python Programming Learning Circle
Aug 28, 2024 · Backend Development

Python Web Scraper for Downloading Online Comics

This article explains how to build a Python script that searches a comic website, extracts chapter links and image URLs using requests and BeautifulSoup, and downloads the images into organized folders with multithreaded support, while outlining required modules and potential improvements.

BeautifulSoupManga DownloaderPython
0 likes · 8 min read
Python Web Scraper for Downloading Online Comics
Python Crawling & Data Mining
Python Crawling & Data Mining
Aug 6, 2024 · Fundamentals

How to Scrape and Download Novels from Biquw.com Using Python

This guide walks you through extracting novel chapters from the Biquw.com website with Python, explaining how to obtain the book ID, retrieve the chapter list, download each chapter using requests and BeautifulSoup, handle common anti‑scraping measures, and organize the downloaded files.

PythonWeb Scrapingnovel-downloader
0 likes · 6 min read
How to Scrape and Download Novels from Biquw.com Using Python
Python Programming Learning Circle
Python Programming Learning Circle
Jul 17, 2024 · Fundamentals

Seven Essential Python Efficiency Tools for Developers

This article introduces seven powerful Python libraries—Pandas, Selenium, Flask, Scrapy, Requests, Faker, and Pillow—explaining their core features, typical use cases, and providing ready‑to‑run code snippets to help developers boost productivity and automate routine tasks.

AutomationData AnalysisLibraries
0 likes · 6 min read
Seven Essential Python Efficiency Tools for Developers
Python Programming Learning Circle
Python Programming Learning Circle
Jul 3, 2024 · Operations

17 Practical Python Scripts for Automating Everyday Tasks

This article presents 17 versatile Python scripts that automate common tasks such as file management, web scraping, email handling, Excel processing, database interaction, system administration, and more, offering code examples and explanations to help developers and analysts improve productivity.

AutomationDatabasePython
0 likes · 28 min read
17 Practical Python Scripts for Automating Everyday Tasks
Python Programming Learning Circle
Python Programming Learning Circle
Jun 15, 2024 · Fundamentals

Six Practical Python Tips to Improve Coding Efficiency

This article presents six practical Python techniques—including exception handling, list comprehensions, regular expressions, web scraping with BeautifulSoup, lambda functions, and simple data visualizations using matplotlib—to help developers write cleaner, more efficient code and enhance their programming workflow.

Data VisualizationException HandlingPython
0 likes · 6 min read
Six Practical Python Tips to Improve Coding Efficiency
php Courses
php Courses
Jun 11, 2024 · Backend Development

Using curl_exec() in PHP to Retrieve Web Content

This article explains how the PHP cURL library, especially the curl_exec() function, can be used to initialize a session, set options, execute HTTP requests, handle errors, and retrieve remote page content efficiently.

HTTP requestWeb Scrapingbackend
0 likes · 4 min read
Using curl_exec() in PHP to Retrieve Web Content