BestHub
Sign in
Tagged articles

BeautifulSoup

72 articles · Page 1 of 1
Python Crawling & Data Mining
Python Crawling & Data Mining
Jul 25, 2026 · Backend Development

Comparing Python’s Four Web‑Scraping Selectors: Regex, BeautifulSoup, XPath, and CSS

This article reviews the four main Python selectors used for web scraping—regular expressions, BeautifulSoup, lxml/XPath, and CSS—detailing their installation, parsing behavior, advantages, drawbacks, and a performance comparison table to help developers choose the most suitable tool for their projects.

BeautifulSoupCSS selectorsPython
0 likes · 8 min read
Comparing Python’s Four Web‑Scraping Selectors: Regex, BeautifulSoup, XPath, and CSS
Model Perspective
Model Perspective
Aug 27, 2025 · Fundamentals

Unlock Python’s Power: 7 Essential Libraries Every Developer Should Master

This article visually explores seven of Python’s most influential third‑party libraries—NumPy, Pandas, Matplotlib, Requests, Scikit‑learn, Flask, and Beautiful Soup—detailing their core features, typical use cases, and sample code to help developers quickly grasp how each tool can accelerate data processing, visualization, web development, and machine‑learning projects.

BeautifulSoupFlaskMatplotlib
0 likes · 12 min read
Unlock Python’s Power: 7 Essential Libraries Every Developer Should Master
DataFunTalk
DataFunTalk
Jun 11, 2025 · Backend Development

Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration

This article provides a comprehensive overview of web‑scraping technologies, compares popular tools such as requests, BeautifulSoup and Selenium, introduces AI‑assisted crawling with DeepSeek, and walks through practical steps for using BrightData’s platform to collect industry data, complete with ready‑to‑run Python code.

BeautifulSoupBrightDataDeepSeek
0 likes · 13 min read
Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration
Python Programming Learning Circle
Python Programming Learning Circle
Jun 7, 2025 · Backend Development

Master Python Web Scraping: From Requests to Selenium and Scrapy

Learn how to efficiently scrape web pages using Python by exploring multiple approaches—including simple requests with BeautifulSoup, fast parsing with lxml, dynamic content extraction with Selenium, and large‑scale crawling with Scrapy—complete with installation steps, code snippets, and detailed explanations.

BeautifulSoupPythonScrapy
0 likes · 10 min read
Master Python Web Scraping: From Requests to Selenium and Scrapy
Python Programming Learning Circle
Python Programming Learning Circle
May 28, 2025 · Backend Development

Using Proxy IPs for Web Scraping with Python: A Practical Guide

This article explains why proxy IPs are essential for reliable web crawling, compares dynamic and static residential proxies, and provides step‑by‑step Python code to scrape product titles, prices and links from Snapdeal while demonstrating how to integrate proxies for improved efficiency and security.

BeautifulSoupdata extractionproxy IP
0 likes · 12 min read
Using Proxy IPs for Web Scraping with Python: A Practical Guide
php Courses
php Courses
May 14, 2025 · Backend Development

Python Advantages for Web Scraping and Core Library Guide

This article outlines Python's advantages for web crawling, introduces core libraries such as Requests, BeautifulSoup, and Scrapy, details a step-by-step development workflow, provides practical code examples for extracting news titles, and highlights important considerations and advanced techniques for robust scraper implementation.

BeautifulSoupPythonScrapy
0 likes · 5 min read
Python Advantages for Web Scraping and Core Library Guide
Code Mala Tang
Code Mala Tang
Apr 19, 2025 · Fundamentals

Master HTML Parsing in Python: BeautifulSoup, lxml, and html.parser Compared

Learn why HTML parsing is essential for web scraping, explore three popular Python libraries—BeautifulSoup, lxml, and the built‑in html.parser—covering installation, core usage, advanced techniques, and a comparative analysis to help you choose the right tool for your project.

BeautifulSoupPythonhtml-parsing
0 likes · 11 min read
Master HTML Parsing in Python: BeautifulSoup, lxml, and html.parser Compared
DataFunSummit
DataFunSummit
Feb 13, 2025 · Big Data

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

This tutorial explains e‑commerce web scraping fundamentals, covering definitions, tool types, data categories, step‑by‑step Python script creation with Requests, BeautifulSoup, and Selenium, provides sample code for Amazon, Walmart, and eBay, discusses challenges like dynamic pages and anti‑scraping measures, and recommends using specialized scraping APIs.

BeautifulSoupBright DataPython
0 likes · 15 min read
E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges
Python Programming Learning Circle
Python Programming Learning Circle
Nov 7, 2024 · Backend Development

11 Efficient Python Web Scraping Tools and a Practical News‑Site Example

This article introduces eleven powerful Python libraries for web scraping—including Requests, BeautifulSoup, Scrapy, Selenium, PyQuery, Lxml, Pandas, Pyppeteer, aiohttp, Faker, and ProxyPool—explains their key features, provides ready‑to‑run code snippets, and demonstrates a real‑world news‑site crawling case study.

BeautifulSoupWeb Scrapingrequests
0 likes · 13 min read
11 Efficient Python Web Scraping Tools and a Practical News‑Site Example
Python Programming Learning Circle
Python Programming Learning Circle
Aug 28, 2024 · Backend Development

Python Web Scraper for Downloading Online Comics

This article explains how to build a Python script that searches a comic website, extracts chapter links and image URLs using requests and BeautifulSoup, and downloads the images into organized folders with multithreaded support, while outlining required modules and potential improvements.

BeautifulSoupManga DownloaderPython
0 likes · 8 min read
Python Web Scraper for Downloading Online Comics
MaGe Linux Operations
MaGe Linux Operations
Mar 15, 2024 · Backend Development

Build a Simple Python Web Scraper to Fetch Recipes in 5 Minutes

This guide walks you through creating a lightweight Python web scraper that fetches recipe data from a cooking website, covering HTTP requests, HTML parsing with BeautifulSoup, extracting titles and images, and wrapping the process into an interactive console application.

BeautifulSoupConsole ApplicationPython
0 likes · 15 min read
Build a Simple Python Web Scraper to Fetch Recipes in 5 Minutes
Python Programming Learning Circle
Python Programming Learning Circle
Dec 1, 2023 · Backend Development

Batch Download Images from a Webpage Using Python

This tutorial explains how to use Python with requests and BeautifulSoup to locate image URLs in a webpage's HTML, extract them via regular expressions, and programmatically download all images to a local folder in an automated batch process.

AutomationBeautifulSoupImage Download
0 likes · 6 min read
Batch Download Images from a Webpage Using Python
Python Programming Learning Circle
Python Programming Learning Circle
Jul 30, 2022 · Backend Development

Python Web Scraping Tutorial: Crawling QDaily, Storing in SQLite, Analyzing Data and Generating a Word Cloud

This tutorial walks through building a simple Python web crawler for the QDaily website, covering target analysis, environment setup, SQLite database creation, data extraction with requests and BeautifulSoup, storing articles and comments, performing basic analysis, and visualizing results with a word cloud.

BeautifulSoupData AnalysisPython
0 likes · 6 min read
Python Web Scraping Tutorial: Crawling QDaily, Storing in SQLite, Analyzing Data and Generating a Word Cloud
Python Programming Learning Circle
Python Programming Learning Circle
Dec 23, 2021 · Backend Development

Extracting Article Content with Python: BeautifulSoup, urllib, and the newspaper Library

This article explains how to automate web article extraction by first using BeautifulSoup and urllib to crawl search results and retrieve page links, then demonstrates a much simpler approach with the Python newspaper library to directly download and parse article text, including full installation and code examples.

BeautifulSouparticle-extractionnewspaper
0 likes · 8 min read
Extracting Article Content with Python: BeautifulSoup, urllib, and the newspaper Library
MaGe Linux Operations
MaGe Linux Operations
Sep 2, 2021 · Backend Development

Build a Python Baidu Baike Crawler: Step-by-Step Guide

This article demonstrates how to create a Python web crawler that fetches Baidu Baike entries, covering the main program structure, URL manager, page downloader, HTML parser using BeautifulSoup, and output generator, with complete code snippets and sample results.

BeautifulSoupPythonWeb Crawler
0 likes · 8 min read
Build a Python Baidu Baike Crawler: Step-by-Step Guide
Python Programming Learning Circle
Python Programming Learning Circle
Aug 20, 2021 · Backend Development

Python Crawler for Scraping Baidu Baike Articles

This article presents a complete Python web crawler example that extracts Baidu Baike entries, detailing the implementation of URL management, page downloading, HTML parsing with BeautifulSoup, data collection, and output generation, along with sample code and usage instructions.

BaikeBeautifulSoupScraping
0 likes · 9 min read
Python Crawler for Scraping Baidu Baike Articles
Python Programming Learning Circle
Python Programming Learning Circle
Jul 14, 2021 · Backend Development

Bypassing Anti‑Scraping Mechanisms: User‑Agent Spoofing and IP Rate Limiting with Python

This article explains how to overcome common anti‑scraping defenses such as identity verification and IP rate limiting by spoofing the User‑Agent header and adding request delays, providing complete Python code examples using requests and BeautifulSoup to scrape Douban's Top 250 movies.

BeautifulSoupIP throttlingUser-Agent
0 likes · 6 min read
Bypassing Anti‑Scraping Mechanisms: User‑Agent Spoofing and IP Rate Limiting with Python
Python Crawling & Data Mining
Python Crawling & Data Mining
Jun 25, 2021 · Backend Development

How to Scrape NBA Player Stats from Hupu and Auto‑Generate Excel Charts with Python

This guide walks you through building a Python web‑scraper that extracts NBA player information from the Hupu website, cleans and visualizes the data, and automatically creates Excel files with embedded line charts, covering URL navigation, data parsing with requests and BeautifulSoup, and chart generation with xlsxwriter.

BeautifulSoupData VisualizationNBA
0 likes · 10 min read
How to Scrape NBA Player Stats from Hupu and Auto‑Generate Excel Charts with Python
MaGe Linux Operations
MaGe Linux Operations
Mar 1, 2021 · Backend Development

Bypass Ant Financial Rental Site Anti‑Scraping with Python Cookies

This tutorial explains how to analyze the Ant Short‑Term Rental website's anti‑scraping mechanisms, extract the required Cookie and User‑Agent headers, and use Python's urllib2 and BeautifulSoup to reliably crawl rental listings, save the data to CSV, and optionally extend the scraper with Selenium.

BeautifulSoupcookiesdata extraction
0 likes · 12 min read
Bypass Ant Financial Rental Site Anti‑Scraping with Python Cookies
Python Crawling & Data Mining
Python Crawling & Data Mining
Jan 15, 2021 · Backend Development

How We Scraped and Analyzed the Global Top 100 Most Beautiful Women with Python

In this article we demonstrate how to use Python’s requests and BeautifulSoup to scrape a static webpage listing the world’s top 100 most beautiful women, process the data with pandas and pyecharts for continent, nationality and profession statistics, and apply Baidu’s face‑recognition API to score each celebrity’s attractiveness.

BeautifulSoupData VisualizationPyecharts
0 likes · 7 min read
How We Scraped and Analyzed the Global Top 100 Most Beautiful Women with Python
Python Programming Learning Circle
Python Programming Learning Circle
Nov 10, 2020 · Backend Development

Detailed Job Description Extraction and Data Cleaning with Python and MongoDB

This article explains how to scrape detailed job description and address information from online job portals, use Python libraries such as requests, BeautifulSoup4, and pymongo for crawling, and then clean and normalize the collected data including publish dates, salaries, and work‑experience levels before storing it in MongoDB.

BeautifulSoupMongoDBPython
0 likes · 8 min read
Detailed Job Description Extraction and Data Cleaning with Python and MongoDB
Python Crawling & Data Mining
Python Crawling & Data Mining
Nov 21, 2019 · Backend Development

Essential Python Web Scraping Libraries Every Developer Should Know

This guide introduces the most important Python libraries for web scraping—including requests, urllib3, Selenium, aiohttp, BeautifulSoup, lxml, pyquery, PyMySQL, PyMongo, and redisdump—explaining their core features, typical use cases, and providing concise code examples to help beginners get started quickly.

BeautifulSoupaiohttplxml
0 likes · 7 min read
Essential Python Web Scraping Libraries Every Developer Should Know
MaGe Linux Operations
MaGe Linux Operations
Jul 19, 2019 · Backend Development

How to Scrape High‑Resolution Images from ColorHub with Python

Learn a step‑by‑step Python solution to locate, download, and store high‑resolution, royalty‑free images from ColorHub by navigating its three‑tier page structure, generating request headers, parsing HTML with BeautifulSoup, and saving files locally, enabling offline PPT creation without copyright concerns.

AutomationBeautifulSoupImage Download
0 likes · 5 min read
How to Scrape High‑Resolution Images from ColorHub with Python
MaGe Linux Operations
MaGe Linux Operations
Jul 2, 2019 · Backend Development

Master Web Scraping with BeautifulSoup: A Complete Python Guide

This tutorial introduces BeautifulSoup, a powerful Python library for parsing HTML and XML, covering installation, basic usage, tag selection, attribute extraction, navigation of parent and sibling nodes, method and CSS selectors, and best‑practice recommendations for efficient web data extraction.

BeautifulSoupPythonWeb Scraping
0 likes · 30 min read
Master Web Scraping with BeautifulSoup: A Complete Python Guide
MaGe Linux Operations
MaGe Linux Operations
Mar 18, 2019 · Backend Development

Master Web Scraping with Beautiful Soup: A Hands‑On Python Guide

Learn how to install and use Beautiful Soup 4 in Python to parse HTML, navigate the document tree, access tags, attributes, and text, and perform powerful searches with methods like find_all, CSS selectors, and traversal techniques for effective web scraping.

BeautifulSoupBeautifulSoup4Python
0 likes · 12 min read
Master Web Scraping with Beautiful Soup: A Hands‑On Python Guide
MaGe Linux Operations
MaGe Linux Operations
Feb 3, 2019 · Backend Development

Master Web Scraping with Beautiful Soup: A Step‑by‑Step Python Guide

This article introduces Beautiful Soup, a Python library for parsing HTML/XML into a navigable tree, covering installation, object initialization, tag navigation, attribute handling, searching techniques like find_all, CSS selectors, and practical code examples for effective web data extraction.

BeautifulSoupPythondata extraction
0 likes · 13 min read
Master Web Scraping with Beautiful Soup: A Step‑by‑Step Python Guide
MaGe Linux Operations
MaGe Linux Operations
Jul 28, 2018 · Backend Development

Master Web Scraping with Beautiful Soup: A Hands‑On Python Guide

This article introduces Beautiful Soup, a Python library for parsing HTML/XML into a navigable tree, covering installation, object initialization, tag and attribute access, tree traversal, searching techniques like find_all, find, CSS selectors, and practical code examples.

BeautifulSoupWeb Scrapingdata extraction
0 likes · 11 min read
Master Web Scraping with Beautiful Soup: A Hands‑On Python Guide
Python Crawling & Data Mining
Python Crawling & Data Mining
Jan 27, 2018 · Backend Development

How to Scrape Real-Time Weather Data with Python and BeautifulSoup

This guide demonstrates how to use Python's BeautifulSoup library to crawl the Green Breath website, extract real‑time weather and PM2.5 information, handle missing data with conditional checks, and display the results directly in the PyCharm console, providing a practical example of web‑scraping for environmental monitoring.

Air QualityBeautifulSoupPython
0 likes · 3 min read
How to Scrape Real-Time Weather Data with Python and BeautifulSoup
21CTO
21CTO
Dec 15, 2017 · Backend Development

Master Web Scraping with Python: Regex, BeautifulSoup & Selenium

This guide demonstrates how to combine Python's regex, BeautifulSoup, and Selenium (including Chrome and headless PhantomJS) for powerful web scraping, covering tag matching, handling Ajax, iFrames, cookie management, and practical code examples for extracting and interacting with dynamic web content.

BeautifulSoupRegexWeb Scraping
0 likes · 10 min read
Master Web Scraping with Python: Regex, BeautifulSoup & Selenium
21CTO
21CTO
Dec 14, 2017 · Backend Development

Master Web Scraping with Python: BeautifulSoup, Selenium & Error Handling

Learn how to scrape static pages, AJAX content, iFrames, and handle cookies using Python libraries such as BeautifulSoup, Selenium, and PhantomJS, while mastering HTTP and URL error handling, CSS‑based element extraction, and practical code examples for robust web data extraction.

BeautifulSoupHTTP Errorshtml-parsing
0 likes · 7 min read
Master Web Scraping with Python: BeautifulSoup, Selenium & Error Handling
MaGe Linux Operations
MaGe Linux Operations
Nov 21, 2017 · Backend Development

How to Scrape All Chinese Stock Data with Python: A Step‑by‑Step Guide

This tutorial explains how to collect the names and trading information of every stock listed on the Shanghai and Shenzhen exchanges using Python 3.5, requests, BeautifulSoup and regular expressions, then store the results in a file through a clear three‑step process of fetching the stock list, retrieving each stock's page, parsing the HTML and writing the data.

BeautifulSoupRegexStock Data
0 likes · 11 min read
How to Scrape All Chinese Stock Data with Python: A Step‑by‑Step Guide
MaGe Linux Operations
MaGe Linux Operations
May 23, 2017 · Backend Development

How to Build a Python Zhihu Web Scraper: Login, User Data, and More

This article walks through building a Python web scraper for Zhihu, covering login simulation, extracting user profiles, answer likers, followers, avatars, and all answers of a question, and storing the collected data in SQLite, while highlighting challenges like captcha and anti‑scraping limits.

BeautifulSoupSQLitedata-extraction
0 likes · 10 min read
How to Build a Python Zhihu Web Scraper: Login, User Data, and More