Tag

anti-scraping

0 views collected around this technical thread.

Python Programming Learning Circle
Python Programming Learning Circle
Oct 11, 2022 · Backend Development

How to Earn Money with Python: Web Scraping, Platforms, and Practical Tips

This guide explains how unemployed developers can use Python, especially web‑scraping techniques, to secure freelance gigs by leveraging various platforms, community groups, and effective order‑taking strategies while warning about common pitfalls and anti‑scraping challenges.

Data ProcessingPythonWeb Scraping
0 likes · 7 min read
How to Earn Money with Python: Web Scraping, Platforms, and Practical Tips
IT Services Circle
IT Services Circle
Mar 26, 2022 · Information Security

Common Reasons Why Your Proxy Fails to Hide Your Web Scraper

The article explains several typical situations—such as not configuring HTTPS proxies, using server IPs, non‑anonymous proxies, polluted IP pools, and lack of HTTP/2 support—that cause websites to easily detect that a request is made through a proxy, even for beginner Python scrapers.

HTTPProxyPython
0 likes · 7 min read
Common Reasons Why Your Proxy Fails to Hide Your Web Scraper
Python Programming Learning Circle
Python Programming Learning Circle
Jul 14, 2021 · Backend Development

Bypassing Anti‑Scraping Mechanisms: User‑Agent Spoofing and IP Rate Limiting with Python

This article explains how to overcome common anti‑scraping defenses such as identity verification and IP rate limiting by spoofing the User‑Agent header and adding request delays, providing complete Python code examples using requests and BeautifulSoup to scrape Douban's Top 250 movies.

BeautifulSoupIP throttlingPython
0 likes · 6 min read
Bypassing Anti‑Scraping Mechanisms: User‑Agent Spoofing and IP Rate Limiting with Python
Python Programming Learning Circle
Python Programming Learning Circle
Dec 25, 2020 · Backend Development

Bypassing Anti‑Scraping Measures on Mayi Short‑Rent Site Using Cookies and BeautifulSoup

This tutorial explains how to analyze the Mayi short‑rent website, overcome its anti‑scraping defenses by setting appropriate Cookie and User‑Agent headers, and use Python's urllib2 and BeautifulSoup to extract rental details, store them in CSV, and optionally employ Selenium.

BeautifulSoupPythonWeb Scraping
0 likes · 8 min read
Bypassing Anti‑Scraping Measures on Mayi Short‑Rent Site Using Cookies and BeautifulSoup
Python Programming Learning Circle
Python Programming Learning Circle
Dec 17, 2020 · Backend Development

Request Header Spoofing and Anti‑Anti‑Scraping Techniques for Web Crawlers

This article explains how to disguise a web crawler's identity by customizing request headers, managing request frequency with sleep and proxy settings, and tackling common anti‑scraping mechanisms such as captchas, dynamic loading, and encrypted content using tools like Selenium.

Seleniumanti-scrapingproxies
0 likes · 6 min read
Request Header Spoofing and Anti‑Anti‑Scraping Techniques for Web Crawlers
Python Programming Learning Circle
Python Programming Learning Circle
Jun 20, 2020 · Information Security

Bypassing Implicit Style‑CSS Anti‑Scraping: Analysis and Restoration of Obfuscated Content

This article explains how many Chinese web sites use hidden CSS ::before content to hide characters, shows how to locate the relevant network request, decode the span class mappings from obfuscated JavaScript, and restore the original text for successful web scraping.

CSSJavaScriptanti-scraping
0 likes · 10 min read
Bypassing Implicit Style‑CSS Anti‑Scraping: Analysis and Restoration of Obfuscated Content
Python Programming Learning Circle
Python Programming Learning Circle
Jun 6, 2020 · Information Security

Understanding CSS Sprites and Techniques to Bypass Sprite‑Based Anti‑Scraping

This article explains the concept and benefits of CSS sprites, analyzes their drawbacks for web performance and security, and provides a step‑by‑step Python‑based method—including code snippets—to extract and sum numbers hidden behind sprite images used as an anti‑scraping measure.

CSSFront-endPython
0 likes · 9 min read
Understanding CSS Sprites and Techniques to Bypass Sprite‑Based Anti‑Scraping
Sohu Tech Products
Sohu Tech Products
Mar 25, 2020 · Information Security

Designing Anti‑Scraping Techniques Using Custom Base64 Encoding

This article explains how to hide real intentions behind visible actions by using text obfuscation and custom Base64‑like encoding to defeat standard web scrapers, detailing the underlying principles, decoding challenges, and Python implementations of a flexible Custom64 encoder.

Base64PythonWeb Security
0 likes · 10 min read
Designing Anti‑Scraping Techniques Using Custom Base64 Encoding
Python Programming Learning Circle
Python Programming Learning Circle
Oct 19, 2019 · Backend Development

How to Bypass Anti‑Scraping Measures: User‑Agent, Cookies & Proxies

This guide explains practical techniques such as faking User‑Agent headers, rotating cookies, adding random delays, and using proxy pools to prevent IP bans while crawling large amounts of data from websites with anti‑scraping defenses.

Web Scrapinganti-scrapingcookies
0 likes · 4 min read
How to Bypass Anti‑Scraping Measures: User‑Agent, Cookies & Proxies
JD Tech
JD Tech
Sep 7, 2018 · Information Security

Big Data and AI Security Insights from ISC 2018 Conference

The ISC 2018 conference highlighted the growing importance of big data and artificial intelligence security, presenting JD's research on anti‑scraping techniques, AI‑driven defenses against black‑market attacks, and a service‑oriented approach to protecting user data across enterprises.

AI securityBig Dataanti-scraping
0 likes · 5 min read
Big Data and AI Security Insights from ISC 2018 Conference