Getting Started with requests-html: Installation, Basic Usage, Advanced Features, and Web Scraping Examples

This article introduces the Python requests-html library, covering its installation, basic operations such as fetching pages, extracting links and elements, advanced capabilities like JavaScript rendering, pagination, custom requests, and provides practical web‑scraping examples for sites like Jianshu and Tianya.

automationhtml-parsingrequests-html

0 likes · 16 min read

Getting Started with requests-html: Installation, Basic Usage, Advanced Features, and Web Scraping Examples

Python Crawling & Data Mining

Sep 22, 2024 · Backend Development

Master Python Web Scraping with requests_html: A Step-by-Step Guide

Learn how to overcome anti‑scraping defenses by using Python's requests_html library to fetch and parse dynamic web pages, with a complete code example that extracts company names from a target site, plus tips on handling cookies, headers, and Unicode decoding.

anti-scrapingdata-miningrequests-html

0 likes · 6 min read

Master Python Web Scraping with requests_html: A Step-by-Step Guide

Python Crawling & Data Mining

Dec 8, 2023 · Big Data

Scrape Shanghai Gold Exchange Data with Python: Step‑by‑Step Guide

This article walks through a step‑by‑step Python solution for scraping Shanghai Gold Exchange transaction data, using requests_html to collect URLs, pandas to parse HTML tables, and finally exporting the aggregated results to Excel, while also offering practical tips for sharing code and debugging.

Pythondata-miningpandas

0 likes · 5 min read

Scrape Shanghai Gold Exchange Data with Python: Step‑by‑Step Guide

Python Crawling & Data Mining

Feb 2, 2022 · Backend Development

Bypass SVG Anti‑Scraping and Extract Data with Selenium and requests‑html

This article explains how to scrape data protected by SVG background‑image anti‑scraping by using Selenium to retrieve the SVG URL, parsing the SVG with requests‑html to map background offsets to characters, replacing SVG nodes with text, and finally extracting structured information such as phone numbers and reviews.

Data ExtractionSVGSelenium

0 likes · 11 min read

Bypass SVG Anti‑Scraping and Extract Data with Selenium and requests‑html

Python Programming Learning Circle

Mar 31, 2021 · Backend Development

Getting Started with requests-html: Installation, Basic Usage, and Advanced Features

This article introduces the Python requests-html library, covering its installation, basic operations such as fetching pages, extracting links and elements with CSS and XPath selectors, advanced capabilities like JavaScript rendering, pagination handling, custom request options, and practical web‑scraping examples.

JavaScript renderingPythonrequests-html

0 likes · 16 min read

Getting Started with requests-html: Installation, Basic Usage, and Advanced Features

MaGe Linux Operations

Sep 18, 2020 · Backend Development

Master Web Scraping with Python requests‑html: Install, Basics & Advanced Tips

This tutorial introduces Python's requests‑html library, covering installation, basic page fetching, link extraction, element selection with CSS and XPath, rendering JavaScript, pagination, direct HTML usage, custom request options, form login, and practical crawling examples.

PythonTutorialhtml-parsing

0 likes · 17 min read

Master Web Scraping with Python requests‑html: Install, Basics & Advanced Tips

MaGe Linux Operations

May 12, 2018 · Backend Development

Unlock Seamless Web Scraping with Requests‑HTML: A Complete Python Guide

The article introduces the Requests‑HTML library—a powerful Python tool that merges HTTP downloading and HTML parsing, outlines its key features such as JavaScript support and built‑in selectors, and provides step‑by‑step code examples for quick, efficient web scraping.

Pythonasynciohtml-parsing

0 likes · 6 min read

Unlock Seamless Web Scraping with Requests‑HTML: A Complete Python Guide