BestHub
Sign in
Tagged articles
143 articles
Page 1 of 2
Su San Talks Tech
Su San Talks Tech
Mar 2, 2026 · Backend Development

How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping

This guide explains how OpenClaw’s built‑in browser feature and the optional Browser Relay extension enable effortless data extraction from both public and login‑required websites, providing step‑by‑step installation commands, Chrome extension loading instructions, and a practical example of scraping top posts from a social platform.

Browser AutomationChrome ExtensionData Extraction
0 likes · 6 min read
How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping
php Courses
php Courses
Dec 10, 2025 · Backend Development

Master PHP’s array_column(): Extract Values from Multidimensional Arrays Easily

This guide explains PHP’s array_column() function—available since version 5.5—detailing its syntax, parameters, return value, and practical code examples that show how to extract specific keys from multidimensional arrays and optionally use another key as the resulting array’s index.

Data ExtractionPHParray_column
0 likes · 4 min read
Master PHP’s array_column(): Extract Values from Multidimensional Arrays Easily
Python Programming Learning Circle
Python Programming Learning Circle
Oct 16, 2025 · Backend Development

Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation

This article introduces DrissionPage, a Python library that merges Selenium and Requests, explains its three page objects, highlights seamless mode switching, built‑in utilities, API changes in version 4.0, and provides practical code examples for web automation, data crawling, and testing.

Browser AutomationData ExtractionSelenium
0 likes · 14 min read
Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation
Dunmao Tech Hub
Dunmao Tech Hub
Aug 9, 2025 · Fundamentals

Convert DOC to DOCX and Extract Tables with Python & LibreOffice

This tutorial explains why converting legacy .doc files to .docx is necessary, shows how to install LibreOffice and python-docx, provides a Python script to perform the conversion via LibreOffice's command‑line interface, and demonstrates reading and modifying tables in the resulting .docx files.

Data Extractiondocxlibreoffice
0 likes · 8 min read
Convert DOC to DOCX and Extract Tables with Python & LibreOffice
Python Programming Learning Circle
Python Programming Learning Circle
May 28, 2025 · Backend Development

Using Proxy IPs for Web Scraping with Python: A Practical Guide

This article explains why proxy IPs are essential for reliable web crawling, compares dynamic and static residential proxies, and provides step‑by‑step Python code to scrape product titles, prices and links from Snapdeal while demonstrating how to integrate proxies for improved efficiency and security.

Data Extractionbeautifulsoupproxy IP
0 likes · 12 min read
Using Proxy IPs for Web Scraping with Python: A Practical Guide
php Courses
php Courses
May 14, 2025 · Backend Development

Python Advantages for Web Scraping and Core Library Guide

This article outlines Python's advantages for web crawling, introduces core libraries such as Requests, BeautifulSoup, and Scrapy, details a step-by-step development workflow, provides practical code examples for extracting news titles, and highlights important considerations and advanced techniques for robust scraper implementation.

Data ExtractionPythonScrapy
0 likes · 5 min read
Python Advantages for Web Scraping and Core Library Guide
DataFunSummit
DataFunSummit
Feb 13, 2025 · Big Data

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

This tutorial explains e‑commerce web scraping fundamentals, covering definitions, tool types, data categories, step‑by‑step Python script creation with Requests, BeautifulSoup, and Selenium, provides sample code for Amazon, Walmart, and eBay, discusses challenges like dynamic pages and anti‑scraping measures, and recommends using specialized scraping APIs.

Bright DataData ExtractionPython
0 likes · 15 min read
E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges
php Courses
php Courses
Dec 26, 2024 · Backend Development

Python vs PHP for Web Scraping: A Comparative Guide

This article compares Python and PHP for web scraping, outlining each language's strengths, ecosystem, performance, learning curve, and community support to help readers decide which tool best fits their project requirements and experience level.

Data ExtractionWeb Scrapingprogramming comparison
0 likes · 9 min read
Python vs PHP for Web Scraping: A Comparative Guide
php Courses
php Courses
Dec 9, 2024 · Backend Development

Extracting Values from a Two-Dimensional PHP Array by ID

This article demonstrates how to create a reusable PHP function that searches a two‑dimensional array for a specific id and returns the value of a given key, such as the title, using a simple loop and conditional check.

ArrayBackendData Extraction
0 likes · 2 min read
Extracting Values from a Two-Dimensional PHP Array by ID
Java Tech Enthusiast
Java Tech Enthusiast
Nov 23, 2024 · Operations

Using jq to Extract JSON Data: A Simple Tutorial

This tutorial shows how to use the jq command‑line tool to extract each user's name from a JSON array, illustrating simple commands such as jq '.users[].name' data.json, describing the author's move from manual IDE methods, and emphasizing the importance of knowing jq exists and leveraging AI to generate scripts from plain‑language descriptions.

AI assistanceData ExtractionJSON
0 likes · 3 min read
Using jq to Extract JSON Data: A Simple Tutorial
Test Development Learning Exchange
Test Development Learning Exchange
May 28, 2024 · Fundamentals

Using Regular Expressions for API Automation Testing: 10 Practical Examples

This article demonstrates how regular expressions can be applied in API automation testing through ten practical Python examples that extract URLs, validate emails, parse JSON, match phone numbers, detect sensitive words, retrieve HTTP status codes, verify dates, parse query parameters, enforce password strength, and extract XML tag content.

Data Extractionvalidation
0 likes · 6 min read
Using Regular Expressions for API Automation Testing: 10 Practical Examples
Liangxu Linux
Liangxu Linux
Mar 12, 2024 · Fundamentals

Mastering awk: Essential Syntax, Patterns, and Real-World Examples

This guide introduces awk as a powerful text‑processing tool, covering its basic syntax, field separators, pattern matching, arithmetic operations, built‑in variables, custom functions, control flow, and how to process both files and command output with clear code examples.

Data ExtractionShell scriptingcommand-line
0 likes · 6 min read
Mastering awk: Essential Syntax, Patterns, and Real-World Examples
Python Crawling & Data Mining
Python Crawling & Data Mining
Dec 27, 2023 · Backend Development

How to Scrape ETF Data with Python: Step-by-Step Code and Tips

This article walks through retrieving ETF fund codes and names from Eastmoney using Python's requests and pandas, explains constructing the correct URLs, handling pagination, cleaning the JSON response, and provides complete sample scripts, while also highlighting a simpler solution and recommending a data‑collection platform.

Data ExtractionETFWeb Scraping
0 likes · 8 min read
How to Scrape ETF Data with Python: Step-by-Step Code and Tips
Java Architecture Diary
Java Architecture Diary
Dec 5, 2023 · Backend Development

Master JSON Pointer with Jackson: Syntax, Examples, and Java Code

This article explains the JSON Pointer syntax, provides encoding rules and practical examples, demonstrates how to extract values from complex JSON structures using Jackson's JsonUtil in Java, and shows how to map nodes to Java beans, offering a concise guide for developers.

Data ExtractionJSONJSON Pointer
0 likes · 6 min read
Master JSON Pointer with Jackson: Syntax, Examples, and Java Code
Python Programming Learning Circle
Python Programming Learning Circle
Nov 3, 2023 · Backend Development

Curated Collection of Python Web Scraping Tools and Tutorials

This article compiles a variety of Python web‑scraping utilities—including file download assistants, video downloaders, proxy IP pool builders, captcha bypass scripts, and game data extractors—providing descriptions, installation commands, usage examples, download links, and reference links for each tool.

BackendData ExtractionWeb Scraping
0 likes · 6 min read
Curated Collection of Python Web Scraping Tools and Tutorials
Test Development Learning Exchange
Test Development Learning Exchange
Jul 6, 2023 · Backend Development

Scrapy Framework Overview and Usage Guide

Scrapy is a powerful Python-based web scraping framework designed for large-scale and complex website data extraction. It offers high-level abstractions, built-in data extraction tools using XPath and CSS selectors, asynchronous processing for parallel requests, and flexible pipelines for data storage, making it ideal for efficient and scalable web scraping projects.

Backend DevelopmentData ExtractionPython
0 likes · 5 min read
Scrapy Framework Overview and Usage Guide
Python Crawling & Data Mining
Python Crawling & Data Mining
Jul 3, 2023 · Backend Development

How to Fix Common Python Web‑Scraping Issues with Ready‑to‑Use Code

This article walks through a real Python web‑scraping problem, explains why missing request headers cause failures, and provides complete, runnable code snippets—including header setup, cookie handling, and request loops—to retrieve stock data from Xueqiu, followed by a concise summary of the solution.

Data ExtractionWeb Scrapingrequests
0 likes · 5 min read
How to Fix Common Python Web‑Scraping Issues with Ready‑to‑Use Code
Architecture and Beyond
Architecture and Beyond
Jul 1, 2023 · Industry Insights

Web Crawlers Unveiled: History, Value, and How to Tackle Their Challenges

This article traces the development of web crawlers from their 1990s origins to modern implementations, examines their multifaceted value in search, data analysis, and archiving, outlines technical, ethical, and legal challenges for both crawler creators and target sites, and presents practical strategies to mitigate malicious crawling.

Data ExtractionSecurityWeb Crawling
0 likes · 24 min read
Web Crawlers Unveiled: History, Value, and How to Tackle Their Challenges
Laravel Tech Community
Laravel Tech Community
Jun 28, 2023 · Fundamentals

Comprehensive Guide to Using AWK for Text Processing

This article provides a thorough introduction to the AWK language, covering its basic syntax, common usage patterns, operators, built‑in variables, and numerous practical examples for processing and extracting data from text files.

Data ExtractionScriptingUnix
0 likes · 10 min read
Comprehensive Guide to Using AWK for Text Processing
Python Crawling & Data Mining
Python Crawling & Data Mining
Jun 13, 2023 · Backend Development

How to Scrape 51Job Listings with Python: A Complete Guide

This article walks through a Python-based web scraper that extracts job listings from 51job.com, detailing required header and cookie settings, pagination logic, data parsing, and CSV output, and includes full code snippets and tips for handling site changes and further data analysis.

Data ExtractionJob DataWeb Scraping
0 likes · 9 min read
How to Scrape 51Job Listings with Python: A Complete Guide
Python Crawling & Data Mining
Python Crawling & Data Mining
May 1, 2023 · Backend Development

Scrape Shandong Government Data with Python: Full Code Walkthrough

This article walks through a real‑world Python web‑scraping case, showing how to retrieve disease‑pest data from the Shandong government site using custom headers, cookies, and request parameters, and provides two complete code examples that successfully return the desired JSON results.

Backend DevelopmentData ExtractionWeb Scraping
0 likes · 6 min read
Scrape Shandong Government Data with Python: Full Code Walkthrough
Python Crawling & Data Mining
Python Crawling & Data Mining
Mar 17, 2023 · Backend Development

Build a Python Stock Data Scraper with Requests and Pandas

This article walks through building a Python web scraper that fetches stock trading data using the requests library and pandas, showing the complete code, how to set headers and cookies, and the resulting DataFrame, while highlighting the limits of relying solely on ChatGPT‑generated snippets.

APIData ExtractionWeb Scraping
0 likes · 4 min read
Build a Python Stock Data Scraper with Requests and Pandas
Python Crawling & Data Mining
Python Crawling & Data Mining
Feb 13, 2023 · Backend Development

Master Python Web Scraping & Data Extraction with Requests, lxml, pandas

This article walks through a Python web‑scraping solution that fetches GDP data from a website using the requests library, parses HTML with lxml, and demonstrates two approaches—manual XPath extraction and a streamlined pandas.read_html method—while providing complete code snippets and tips for handling pagination and data storage.

Data ExtractionWeb Scrapinglxml
0 likes · 6 min read
Master Python Web Scraping & Data Extraction with Requests, lxml, pandas
Python Programming Learning Circle
Python Programming Learning Circle
Dec 28, 2022 · Fundamentals

13 Advanced Python Scripts for Everyday Tasks

This article presents thirteen advanced Python scripts covering tasks such as internet speed testing, Google searching, web automation, lyric retrieval, EXIF extraction, OCR, image cartoonization, recycle bin cleaning, image enhancement, Windows version detection, PDF-to-image conversion, hex-to-RGB conversion, and website status checking, each with code examples.

Advanced ScriptsAutomationData Extraction
0 likes · 10 min read
13 Advanced Python Scripts for Everyday Tasks
Cognitive Technology Team
Cognitive Technology Team
Jun 17, 2022 · Backend Development

Using JsonPath in Java to Simplify JSON Data Access

This article introduces JsonPath as a DSL for reading JSON data in Java, demonstrates Maven integration, provides code examples for extracting and renaming JSON fields, and mentions the jq command‑line tool for additional JSON manipulation.

DSLData ExtractionJSON
0 likes · 4 min read
Using JsonPath in Java to Simplify JSON Data Access
IT Services Circle
IT Services Circle
Mar 7, 2022 · Big Data

Awesome Web Scraping – A Comprehensive Chinese Collection of Web Scraping Resources

This article introduces the renowned "awesome" GitHub repository, highlights its extensive sub‑lists for various domains, focuses on the awesome‑web‑scraping collection, and presents a newly created Chinese version that aggregates Python, JavaScript, Go, and other language‑specific web‑scraping tools and libraries.

Awesome ListData ExtractionGitHub
0 likes · 4 min read
Awesome Web Scraping – A Comprehensive Chinese Collection of Web Scraping Resources
Python Crawling & Data Mining
Python Crawling & Data Mining
Feb 2, 2022 · Backend Development

Bypass SVG Anti‑Scraping and Extract Data with Selenium and requests‑html

This article explains how to scrape data protected by SVG background‑image anti‑scraping by using Selenium to retrieve the SVG URL, parsing the SVG with requests‑html to map background offsets to characters, replacing SVG nodes with text, and finally extracting structured information such as phone numbers and reviews.

Data ExtractionSVGSelenium
0 likes · 11 min read
Bypass SVG Anti‑Scraping and Extract Data with Selenium and requests‑html
Programmer DD
Programmer DD
Dec 28, 2021 · Backend Development

Master Web Scraping with Java: Getting Started with Jsoup

This article introduces Jsoup, an open‑source Java library for extracting and manipulating HTML, explains its key features such as DOM traversal and CSS selectors, and provides a concise code example that fetches Wikipedia headlines, helping developers automate web data collection.

BackendData ExtractionJava
0 likes · 3 min read
Master Web Scraping with Java: Getting Started with Jsoup
Architect
Architect
Nov 1, 2021 · Fundamentals

Document Rendering and Structured Data Extraction in Baidu Wenku: From Layout Data to Flow Data and Chart Metadata

The article explains Baidu Wenku's document conversion pipeline, detailing how various office formats are transformed into PDF layout data, then into adaptive flow data for mobile devices, and describes the technical methods for extracting structured content and chart metadata from PDFs and OOXML documents.

Baidu WenkuData ExtractionOOXML
0 likes · 11 min read
Document Rendering and Structured Data Extraction in Baidu Wenku: From Layout Data to Flow Data and Chart Metadata
Sohu Tech Products
Sohu Tech Products
Aug 25, 2021 · Backend Development

Scrapy Tutorial: Installation, Project Structure, Basic Usage, and Real‑World Example

This article provides a comprehensive, step‑by‑step guide to the Scrapy web‑crawling framework, covering its core components, installation methods, project layout, spider creation, data extraction techniques, pagination handling, pipeline configuration, and how to run the crawler to collect and store data.

CrawlerData ExtractionPython
0 likes · 13 min read
Scrapy Tutorial: Installation, Project Structure, Basic Usage, and Real‑World Example
Python Crawling & Data Mining
Python Crawling & Data Mining
Mar 11, 2021 · Backend Development

How to Build a Robust Python Web Crawler for Forum Comments with Scrapy & Selenium

This article walks through building a Python web crawler that extracts forum post comments into MongoDB, covering project goals, environment setup, site structure analysis, Scrapy and Selenium integration, data storage design, handling anti‑scraping measures, and performance optimization with multithreading.

CrawlerData ExtractionMongoDB
0 likes · 13 min read
How to Build a Robust Python Web Crawler for Forum Comments with Scrapy & Selenium
MaGe Linux Operations
MaGe Linux Operations
Mar 1, 2021 · Backend Development

Bypass Ant Financial Rental Site Anti‑Scraping with Python Cookies

This tutorial explains how to analyze the Ant Short‑Term Rental website's anti‑scraping mechanisms, extract the required Cookie and User‑Agent headers, and use Python's urllib2 and BeautifulSoup to reliably crawl rental listings, save the data to CSV, and optionally extend the scraper with Selenium.

Data Extractionbeautifulsoupcookies
0 likes · 12 min read
Bypass Ant Financial Rental Site Anti‑Scraping with Python Cookies
Tencent Cloud Developer
Tencent Cloud Developer
Jan 21, 2021 · Big Data

A Beginner's Guide to Using Scrapy for Web Crawling

This beginner‑friendly guide walks readers through installing Scrapy, creating a project and spider, running and debugging crawlers, implementing parsing with CSS/XPath, and overcoming common hurdles such as JavaScript rendering, user‑agent spoofing, and proxy rotation via configurable middlewares, enabling quick start of web‑crawling projects.

Data ExtractionProxyPython
0 likes · 13 min read
A Beginner's Guide to Using Scrapy for Web Crawling
Python Crawling & Data Mining
Python Crawling & Data Mining
Oct 24, 2020 · Backend Development

Master Scrapy: Extract Likes, Comments, and Content with XPath

This article continues a Scrapy tutorial by showing how to extract like counts, comment counts, and full article content using XPath selectors, regular expressions, and debugging techniques, providing step‑by‑step code examples and screenshots to help Python developers automate web data collection.

Data ExtractionPythonScrapy
0 likes · 6 min read
Master Scrapy: Extract Likes, Comments, and Content with XPath
MaGe Linux Operations
MaGe Linux Operations
Sep 28, 2020 · Backend Development

Build a Scalable Python Web Scraper for 3000+ Companies

This article walks through creating a Python web scraper that extracts financial data for over three thousand listed companies, starting from a simple pandas script and progressively adding error handling, MySQL storage, and multiprocessing to build a robust, production‑ready tool.

Data ExtractionPythonWeb Scraping
0 likes · 7 min read
Build a Scalable Python Web Scraper for 3000+ Companies
FunTester
FunTester
Aug 14, 2020 · Backend Development

Mastering JsonPath: Extract Ordered Objects and Slices from JSON in Java

This tutorial demonstrates how to use JsonPath in Java to retrieve specific items and slices from JSON arrays, covering zero‑based indexing, negative indices for reverse lookup, Python‑style slice syntax, and provides complete code examples with console output.

Data Extraction
0 likes · 5 min read
Mastering JsonPath: Extract Ordered Objects and Slices from JSON in Java
Python Programming Learning Circle
Python Programming Learning Circle
May 18, 2020 · Backend Development

How to Retrieve Binance Trade Data with Python: A Step-by-Step Guide

This article explains why accurate trade data is essential for strategy backtesting, why Binance is chosen, and provides a detailed Python workflow—including argument parsing, using the Binance aggTrades endpoint, handling pagination with from_id, cleaning the resulting DataFrame, saving to CSV, and validating the data integrity.

Binance APICSVData Extraction
0 likes · 7 min read
How to Retrieve Binance Trade Data with Python: A Step-by-Step Guide