BestHub
Sign in
Tagged articles
466 articles
Page 1 of 5
AI Architecture Path
AI Architecture Path
May 6, 2026 · Backend Development

Scrapling: Self‑Healing Web Scraper That Bypasses Cloudflare and Is 784× Faster Than BS4

Scrapling is an open‑source, adaptive web‑scraping framework that automatically tracks element changes, bypasses Cloudflare and other anti‑scraping defenses, offers multiple fetchers (including stealth mode), and delivers extraction speeds up to 784× faster than BeautifulSoup (BS4) while supporting concurrency, AI integration, and easy CLI usage.

PythonScraplingWeb Scraping
0 likes · 16 min read
Scrapling: Self‑Healing Web Scraper That Bypasses Cloudflare and Is 784× Faster Than BS4
AI Engineering
AI Engineering
May 3, 2026 · Backend Development

Rust‑Based Headless Browser Uses 85% Less Resources Than Chrome

Obscura, an open‑source Rust headless browser designed for AI agents and large‑scale crawling, cuts memory usage by 85%, reduces binary size, speeds page loads to 85 ms, starts instantly, includes strong anti‑detection features, and works with the Chrome DevTools Protocol as a drop‑in replacement for Headless Chrome.

CDPHeadless BrowserObscura
0 likes · 5 min read
Rust‑Based Headless Browser Uses 85% Less Resources Than Chrome
Wu Shixiong's Large Model Academy
Wu Shixiong's Large Model Academy
Apr 14, 2026 · Artificial Intelligence

Designing High‑Quality Tools for Deep Research Agents: From Search to Python Execution

This article explains how to turn simple API calls into robust, noise‑filtering tools—Search, Visit, Scholar, and Python—by adding domain blacklists, relevance scoring, query‑driven extraction, safety sandboxes, and a unified registry, ultimately boosting the success rate of LLM‑driven research agents.

AI AgentsLLM safetyReact
0 likes · 32 min read
Designing High‑Quality Tools for Deep Research Agents: From Search to Python Execution
AI Architecture Path
AI Architecture Path
Mar 22, 2026 · Backend Development

How Lightpanda Delivers 11× Faster Headless Browsing with 1/9 Memory Usage

Lightpanda, an open‑source headless browser written in Zig, claims up to eleven times faster execution and one‑ninth the memory of Chrome, offering a lightweight architecture for large‑scale web scraping, AI agent testing, and automation with detailed benchmarks, feature lists, usage commands, and known limitations.

Headless BrowserWeb ScrapingZig
0 likes · 9 min read
How Lightpanda Delivers 11× Faster Headless Browsing with 1/9 Memory Usage
Code Wrench
Code Wrench
Mar 16, 2026 · Backend Development

Mastering Chromedp: Go‑Based Chrome Automation for Real‑World Use Cases

This article provides a comprehensive guide to using the Go library chromedp for Chrome DevTools Protocol‑driven browser automation, covering its core architecture, practical techniques for navigation, network interception, anti‑bot evasion, performance tuning, and production‑grade recommendations with full code examples.

Browser AutomationChrome DevTools ProtocolGo
0 likes · 10 min read
Mastering Chromedp: Go‑Based Chrome Automation for Real‑World Use Cases
Su San Talks Tech
Su San Talks Tech
Mar 2, 2026 · Backend Development

How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping

This guide explains how OpenClaw’s built‑in browser feature and the optional Browser Relay extension enable effortless data extraction from both public and login‑required websites, providing step‑by‑step installation commands, Chrome extension loading instructions, and a practical example of scraping top posts from a social platform.

Browser AutomationChrome ExtensionData Extraction
0 likes · 6 min read
How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping
Old Meng AI Explorer
Old Meng AI Explorer
Jan 22, 2026 · Industry Insights

Three Must‑Try Open‑Source AI Tools for Scraping, PPT Creation, and Video Generation

This article reviews three recently popular open‑source AI tools—Spider_XHS for Xiaohongshu data scraping, PPTAgent for one‑click presentation generation, and Code2Video for code‑driven video creation—detailing their key features, quick‑start instructions, and GitHub repositories, all free and ready to deploy.

AI toolsPPT generationWeb Scraping
0 likes · 7 min read
Three Must‑Try Open‑Source AI Tools for Scraping, PPT Creation, and Video Generation
AI Insight Log
AI Insight Log
Jan 4, 2026 · Artificial Intelligence

How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt

The article examines the open‑source ai‑goofish‑monitor project, which combines Playwright‑driven browsing with large‑language‑model analysis to continuously scan Xianyu listings, filter out junk, and highlight high‑quality items, while also discussing its AI‑generated code, benefits, limitations, and security risks.

AILLMPlaywright
0 likes · 7 min read
How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt
Open Source Tech Hub
Open Source Tech Hub
Dec 28, 2025 · Backend Development

Automate Chrome with PHP: Install and Use ChromePHP for Screenshots, PDFs, and More

ChromePHP is a PHP library that integrates the Chrome DevTools protocol, allowing developers to control Chrome or Chromium directly from PHP code for tasks such as navigation, JavaScript execution, screenshots, PDF generation, and input simulation, with simple Composer installation and flexible configuration.

Chrome automationChromePHPDevTools
0 likes · 3 min read
Automate Chrome with PHP: Install and Use ChromePHP for Screenshots, PDFs, and More
Old Meng AI Explorer
Old Meng AI Explorer
Dec 10, 2025 · Operations

How Spider_XHS Turns Xiaohongshu Data Collection into a 10× Efficiency Boost

Spider_XHS is an open‑source Xiaohongshu crawler that automates note, user, comment, and message extraction, offers watermark‑free media downloads, exports structured Excel/JSON data, integrates with the creator platform, and includes proxy and anti‑ban features, enabling marketers and researchers to cut weeks of manual work into hours.

Web ScrapingXiaohongshuautomation
0 likes · 10 min read
How Spider_XHS Turns Xiaohongshu Data Collection into a 10× Efficiency Boost
Python Programming Learning Circle
Python Programming Learning Circle
Oct 16, 2025 · Backend Development

Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation

This article introduces DrissionPage, a Python library that merges Selenium and Requests, explains its three page objects, highlights seamless mode switching, built‑in utilities, API changes in version 4.0, and provides practical code examples for web automation, data crawling, and testing.

Browser AutomationData ExtractionSelenium
0 likes · 14 min read
Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 5, 2025 · Artificial Intelligence

How Browser-Use Leverages LLMs to Transform Browser Automation

This article explores Browser-Use, an AI‑driven browser automation framework that combines large language models, visual perception, and DOM analysis to enable intelligent, multi‑step web tasks such as registration, price comparison, form filling, and monitoring, while detailing its architecture, historical context, core modules, and future challenges.

AI AgentsBrowser AutomationLLM
0 likes · 26 min read
How Browser-Use Leverages LLMs to Transform Browser Automation
37 Interactive Technology Team
37 Interactive Technology Team
Sep 3, 2025 · Artificial Intelligence

How AI is Revolutionizing Web Scraping: Tools, Techniques, and Best Practices

Discover how AI, especially large language models, transforms traditional web scraping by introducing semantic understanding, dynamic adaptability, and automated extraction, with in‑depth reviews of emerging tools like Crawl4AI and Browser‑use, practical code examples, best‑practice guidelines, and deployment tips for modern data collection.

AIBrowser UseCrawl4AI
0 likes · 17 min read
How AI is Revolutionizing Web Scraping: Tools, Techniques, and Best Practices
21CTO
21CTO
Aug 19, 2025 · Artificial Intelligence

How Crawl4AI Transforms Web Scraping with AI‑Powered Automation

Crawl4AI is an open‑source AI agent that automates web crawling and data extraction, offering free usage, intelligent parsing, structured JSON/Markdown output, and versatile features like scrolling, multi‑URL scraping, media and metadata extraction, all demonstrated through step‑by‑step Python examples and integration with AI agents.

AI AgentsCrawl4AILLM extraction
0 likes · 8 min read
How Crawl4AI Transforms Web Scraping with AI‑Powered Automation
Python Programming Learning Circle
Python Programming Learning Circle
Jul 24, 2025 · Backend Development

Boost Your Python Productivity: 8 Must‑Use Libraries for Faster Development

Discover eight essential Python libraries—Rich, Typer, Pendulum, Pydantic, Faker, tqdm, Requests‑HTML, and Loguru—that transform logging, CLI creation, date handling, data validation, mock data generation, progress tracking, web scraping, and logging, showing how leveraging existing tools can dramatically increase development efficiency.

PythonWeb Scrapingdata validation
0 likes · 8 min read
Boost Your Python Productivity: 8 Must‑Use Libraries for Faster Development
DevOps
DevOps
Jun 16, 2025 · Artificial Intelligence

Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code

This article reviews six open‑source MCP servers—Bright Data, Graphiti, GitIngest, Terminal, Code Executor, and MindsDB—showing how each extends large language models with web scraping, long‑term memory, code navigation, command‑line control, sandboxed Python execution, and multi‑source data integration, complete with practical code examples.

AI toolsCode ExecutionKnowledge Graph
0 likes · 9 min read
Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code
DataFunTalk
DataFunTalk
Jun 11, 2025 · Backend Development

Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration

This article provides a comprehensive overview of web‑scraping technologies, compares popular tools such as requests, BeautifulSoup and Selenium, introduces AI‑assisted crawling with DeepSeek, and walks through practical steps for using BrightData’s platform to collect industry data, complete with ready‑to‑run Python code.

BrightDataDeepSeekPython
0 likes · 13 min read
Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration
Python Programming Learning Circle
Python Programming Learning Circle
Jun 7, 2025 · Backend Development

Master Python Web Scraping: From Requests to Selenium and Scrapy

Learn how to efficiently scrape web pages using Python by exploring multiple approaches—including simple requests with BeautifulSoup, fast parsing with lxml, dynamic content extraction with Selenium, and large‑scale crawling with Scrapy—complete with installation steps, code snippets, and detailed explanations.

PythonScrapySelenium
0 likes · 10 min read
Master Python Web Scraping: From Requests to Selenium and Scrapy
php Courses
php Courses
May 14, 2025 · Backend Development

Python Advantages for Web Scraping and Core Library Guide

This article outlines Python's advantages for web crawling, introduces core libraries such as Requests, BeautifulSoup, and Scrapy, details a step-by-step development workflow, provides practical code examples for extracting news titles, and highlights important considerations and advanced techniques for robust scraper implementation.

Data ExtractionPythonScrapy
0 likes · 5 min read
Python Advantages for Web Scraping and Core Library Guide
Python Programming Learning Circle
Python Programming Learning Circle
Apr 30, 2025 · Backend Development

Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib

This article demonstrates how to use Python's requests and BeautifulSoup libraries to scrape current and 14‑day weather data from China Weather, store the results in CSV files, and perform comprehensive visual analysis—including temperature, humidity, AQI, wind direction, and forecast charts—using pandas, numpy, and matplotlib.

Data visualizationWeather AnalysisWeb Scraping
0 likes · 26 min read
Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib
Python Programming Learning Circle
Python Programming Learning Circle
Apr 1, 2025 · Backend Development

Implementing Local QR Code Login for QQ Music Using Python

This tutorial walks through building a Python script that fetches a QQ Music QR code, displays it locally, monitors its status, extracts required encrypted parameters from cookies, and completes the login process by repeatedly polling the authentication endpoint to obtain a session.

QR loginWeb Scrapingautomation
0 likes · 13 min read
Implementing Local QR Code Login for QQ Music Using Python
DataFunSummit
DataFunSummit
Feb 25, 2025 · Artificial Intelligence

Collecting High-Quality LLM Training Data and Custom Model Training Guide

This article explains what constitutes high‑quality LLM training data, why large datasets are essential, outlines the step‑by‑step process for collecting, preprocessing, and fine‑tuning models, and highlights the best data sources—including web content, books, code repositories, and news—while noting available free datasets.

AILLMWeb Scraping
0 likes · 9 min read
Collecting High-Quality LLM Training Data and Custom Model Training Guide
DataFunSummit
DataFunSummit
Feb 13, 2025 · Big Data

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

This tutorial explains e‑commerce web scraping fundamentals, covering definitions, tool types, data categories, step‑by‑step Python script creation with Requests, BeautifulSoup, and Selenium, provides sample code for Amazon, Walmart, and eBay, discusses challenges like dynamic pages and anti‑scraping measures, and recommends using specialized scraping APIs.

Bright DataData ExtractionPython
0 likes · 15 min read
E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges
php Courses
php Courses
Dec 26, 2024 · Backend Development

Python vs PHP for Web Scraping: A Comparative Guide

This article compares Python and PHP for web scraping, outlining each language's strengths, ecosystem, performance, learning curve, and community support to help readers decide which tool best fits their project requirements and experience level.

Data ExtractionWeb Scrapingprogramming comparison
0 likes · 9 min read
Python vs PHP for Web Scraping: A Comparative Guide
Raymond Ops
Raymond Ops
Dec 23, 2024 · Backend Development

Master Python Proxy Techniques for Efficient Web Scraping

This guide explains the essential proxy concepts for Python web scraping, covering what proxies are, common types, how they protect crawlers, practical usage with the requests library, and the role of proxy pools in improving scraping efficiency.

Backend DevelopmentWeb Scraping
0 likes · 7 min read
Master Python Proxy Techniques for Efficient Web Scraping
Python Programming Learning Circle
Python Programming Learning Circle
Nov 25, 2024 · Backend Development

Python Web Scraping of Fund Holdings Data and Analysis Using Requests, Selenium, and MongoDB

This tutorial demonstrates how to analyze a fund ranking website, construct dynamic URLs, extract six‑digit fund codes, crawl fund holding pages with requests and Selenium, store the results in MongoDB, and finally process the data to identify the most frequently held stocks across thousands of funds.

MongoDBSeleniumWeb Scraping
0 likes · 13 min read
Python Web Scraping of Fund Holdings Data and Analysis Using Requests, Selenium, and MongoDB
Test Development Learning Exchange
Test Development Learning Exchange
Nov 23, 2024 · Operations

Comprehensive Python Automation Scripts for Common Tasks

This article presents a collection of practical Python scripts covering file management, web scraping, email sending, Excel handling, data cleaning, image processing, system monitoring, PDF manipulation, OCR, database interaction, social media posting, testing, and cloud storage, each with clear descriptions and ready‑to‑run code examples.

PythonScriptingWeb Scraping
0 likes · 12 min read
Comprehensive Python Automation Scripts for Common Tasks
Python Crawling & Data Mining
Python Crawling & Data Mining
Nov 8, 2024 · Fundamentals

Extracting Courier Addresses with Python: A Step‑by‑Step Guide

This article walks through a practical Python solution for parsing a list of courier records, extracting province information, grouping entries by location using dictionaries, and optionally applying regular expressions and pandas for further analysis, all illustrated with clear code examples and output screenshots.

DictionariesListsPython
0 likes · 11 min read
Extracting Courier Addresses with Python: A Step‑by‑Step Guide
Python Programming Learning Circle
Python Programming Learning Circle
Nov 7, 2024 · Backend Development

11 Efficient Python Web Scraping Tools and a Practical News‑Site Example

This article introduces eleven powerful Python libraries for web scraping—including Requests, BeautifulSoup, Scrapy, Selenium, PyQuery, Lxml, Pandas, Pyppeteer, aiohttp, Faker, and ProxyPool—explains their key features, provides ready‑to‑run code snippets, and demonstrates a real‑world news‑site crawling case study.

SeleniumWeb Scrapingbeautifulsoup
0 likes · 13 min read
11 Efficient Python Web Scraping Tools and a Practical News‑Site Example
Python Crawling & Data Mining
Python Crawling & Data Mining
Nov 1, 2024 · Backend Development

Fixing Chinese Character Garbling in Python Web Scraping

This article walks through a real‑world Python web‑scraping issue where Chinese characters appear as garbled text, explains why the default encoding fails, and shows how setting the response’s apparent encoding resolves the problem, complete with sample code and practical tips for posting questions.

PythonWeb Scrapingencoding
0 likes · 5 min read
Fixing Chinese Character Garbling in Python Web Scraping
php Courses
php Courses
Oct 21, 2024 · Backend Development

Using PHP get_headers() to Retrieve HTTP Response Headers

This article explains the PHP get_headers() function, its syntax, parameters, and usage examples, demonstrates how to retrieve and display HTTP response headers from a URL, and discusses common application scenarios such as checking remote file existence, obtaining file metadata, and supporting web crawling or monitoring.

BackendHTTPWeb Scraping
0 likes · 4 min read
Using PHP get_headers() to Retrieve HTTP Response Headers
Test Development Learning Exchange
Test Development Learning Exchange
Sep 28, 2024 · Fundamentals

Python Cheat Sheet for File Management, Data Processing, Web Requests, Automation, and Text Manipulation

This article provides a concise Python cheat sheet covering common file and directory operations, CSV/JSON data handling, web request and crawling techniques, automation tasks such as scheduling and email sending, and various string manipulation utilities, each illustrated with ready‑to‑run code snippets.

PythonScriptingWeb Scraping
0 likes · 10 min read
Python Cheat Sheet for File Management, Data Processing, Web Requests, Automation, and Text Manipulation
Python Programming Learning Circle
Python Programming Learning Circle
Aug 28, 2024 · Backend Development

Python Web Scraper for Downloading Online Comics

This article explains how to build a Python script that searches a comic website, extracts chapter links and image URLs using requests and BeautifulSoup, and downloads the images into organized folders with multithreaded support, while outlining required modules and potential improvements.

Manga DownloaderPythonWeb Scraping
0 likes · 8 min read
Python Web Scraper for Downloading Online Comics
Python Crawling & Data Mining
Python Crawling & Data Mining
Aug 6, 2024 · Fundamentals

How to Scrape and Download Novels from Biquw.com Using Python

This guide walks you through extracting novel chapters from the Biquw.com website with Python, explaining how to obtain the book ID, retrieve the chapter list, download each chapter using requests and BeautifulSoup, handle common anti‑scraping measures, and organize the downloaded files.

PythonWeb Scrapingnovel-downloader
0 likes · 6 min read
How to Scrape and Download Novels from Biquw.com Using Python
Python Programming Learning Circle
Python Programming Learning Circle
Jul 17, 2024 · Fundamentals

Seven Essential Python Efficiency Tools for Developers

This article introduces seven powerful Python libraries—Pandas, Selenium, Flask, Scrapy, Requests, Faker, and Pillow—explaining their core features, typical use cases, and providing ready‑to‑run code snippets to help developers boost productivity and automate routine tasks.

PythonWeb Scrapingautomation
0 likes · 6 min read
Seven Essential Python Efficiency Tools for Developers
Python Programming Learning Circle
Python Programming Learning Circle
Jul 3, 2024 · Operations

17 Practical Python Scripts for Automating Everyday Tasks

This article presents 17 versatile Python scripts that automate common tasks such as file management, web scraping, email handling, Excel processing, database interaction, system administration, and more, offering code examples and explanations to help developers and analysts improve productivity.

PythonScriptingWeb Scraping
0 likes · 28 min read
17 Practical Python Scripts for Automating Everyday Tasks
Python Programming Learning Circle
Python Programming Learning Circle
Jun 15, 2024 · Fundamentals

Six Practical Python Tips to Improve Coding Efficiency

This article presents six practical Python techniques—including exception handling, list comprehensions, regular expressions, web scraping with BeautifulSoup, lambda functions, and simple data visualizations using matplotlib—to help developers write cleaner, more efficient code and enhance their programming workflow.

Data visualizationException HandlingPython
0 likes · 6 min read
Six Practical Python Tips to Improve Coding Efficiency
php Courses
php Courses
Jun 11, 2024 · Backend Development

Using curl_exec() in PHP to Retrieve Web Content

This article explains how the PHP cURL library, especially the curl_exec() function, can be used to initialize a session, set options, execute HTTP requests, handle errors, and retrieve remote page content efficiently.

BackendHTTP requestWeb Scraping
0 likes · 4 min read
Using curl_exec() in PHP to Retrieve Web Content
Python Programming Learning Circle
Python Programming Learning Circle
May 30, 2024 · Fundamentals

A Collection of Python Automation Scripts for Media Downloading, Email Alerts, Face Recognition, and More

The article showcases a variety of Python scripts that automate tasks such as subtitle and video downloading, IMDb data extraction, email notifications, PNR status checks, WhatsApp avatar monitoring, face recognition for photo sorting, and a custom DC++ sharing tool, providing code examples and usage details for each utility.

Email NotificationFile DownloaderPython
0 likes · 20 min read
A Collection of Python Automation Scripts for Media Downloading, Email Alerts, Face Recognition, and More
21CTO
21CTO
May 10, 2024 · Artificial Intelligence

Top 10 Python Libraries Every Data Scientist Should Master

This article reviews the ten most essential Python libraries for data science, covering data acquisition, analysis, machine learning, and visualization, and provides concise code examples to help beginners quickly start using tools like Beautiful Soup, NumPy, pandas, scikit‑learn, TensorFlow, Keras, Matplotlib, and seaborn.

NumPyWeb Scrapingdata-science
0 likes · 15 min read
Top 10 Python Libraries Every Data Scientist Should Master
Python Crawling & Data Mining
Python Crawling & Data Mining
Apr 18, 2024 · Backend Development

How to Scrape GDP Data with Python and Save to CSV in Minutes

This article demonstrates how to use Python's requests, lxml, and pandas libraries to crawl GDP data from a website, parse the HTML tables, and efficiently write the extracted rankings, regions, GDP values, and years into a CSV file, providing a complete, runnable example for web scraping beginners.

CSVPythonWeb Scraping
0 likes · 8 min read
How to Scrape GDP Data with Python and Save to CSV in Minutes
Python Programming Learning Circle
Python Programming Learning Circle
Mar 27, 2024 · Operations

9 Practical Python Automation Scenarios with Sample Scripts

This article presents nine practical Python automation scenarios—including file management, web scraping, text processing, email handling, Excel manipulation, database interaction, social media posting, system task scheduling, and image editing—each accompanied by clear script examples and explanations to help readers streamline repetitive tasks efficiently.

EmailPythonScripting
0 likes · 19 min read
9 Practical Python Automation Scenarios with Sample Scripts
MaGe Linux Operations
MaGe Linux Operations
Mar 19, 2024 · Backend Development

Master Python Proxies: 5 Essential Tips for Effective Web Scraping

Learn the core concepts of using proxies in Python web scraping, including what proxies are, common types like anonymous and high‑anonymity, how they protect your crawler, practical implementation with the requests library, and an overview of building a proxy pool for scalable data extraction.

CrawlerPythonWeb Scraping
0 likes · 7 min read
Master Python Proxies: 5 Essential Tips for Effective Web Scraping
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Mar 8, 2024 · Artificial Intelligence

Building a Coze Bot for Weekend Activity Recommendations with Data Retrieval and Custom Plugins

This article walks through creating a Coze chatbot that recommends weekend attractions and events by designing prompt‑based workflows, leveraging official plugins, scraping city‑level data, constructing a knowledge base, handling pagination with code nodes, and finally deploying the bot for real‑time user queries.

AI promptBot DevelopmentCoze
0 likes · 20 min read
Building a Coze Bot for Weekend Activity Recommendations with Data Retrieval and Custom Plugins