Tagged articles
anti‑crawler
12 articles · Page 1 of 1
This guide explains how to use Nginx's built‑in limit_req and limit_conn modules to implement production‑grade rate limiting, covering leaky‑bucket theory, IP and API‑key based limits, burst and nodelay handling, whitelist bypass, custom error responses, dry‑run testing, memory sizing, and the inherent limitations that may require Redis‑Lua or OpenResty extensions.
API protectionLeaky BucketNGINX
0 likes · 21 min read

This comprehensive guide explains why modern web services need rate limiting and anti‑crawler protection, compares token‑bucket and leaky‑bucket algorithms, and provides step‑by‑step Nginx configurations for IP, URI, and geographic throttling, advanced user‑agent filtering, JavaScript challenges, real‑time monitoring, performance tuning, and troubleshooting.
NGINXanti‑crawlerrate limiting
0 likes · 18 min read

This guide walks operations engineers through the principles and practical configurations of Nginx rate limiting and anti‑crawler protection, covering token‑bucket and leaky‑bucket algorithms, IP and URI based limits, geo‑based controls, advanced User‑Agent filtering, JavaScript challenges, monitoring, performance tuning, and troubleshooting.
DevOpsOperationsanti‑crawler
0 likes · 19 min read

Bilibili Comics deploys a three‑layer protection system—stereoscopic tracing, crawler defense, and encrypted channels—that dramatically reduces illegal distribution, boosts creator engagement, and creates a healthier comic ecosystem through real‑time monitoring, automated takedowns, and secure CDN delivery.
BilibiliDigital Rights Managementanti‑crawler
0 likes · 6 min read

This article explains three PHP‑based techniques—captcha verification, encrypted parameter transmission, and dynamic token validation—to protect web login pages from automated crawling and credential‑stealing attacks, while also noting their limitations and implementation details.
CAPTCHADynamic TokenLogin Security
0 likes · 6 min read

The article presents kk-anti-reptile, a Spring Boot‑based anti‑crawler filter that uses a servlet Filter with a rule‑chain architecture, details its system requirements, configuration, integration steps for backend and frontend, and provides code examples and deployment notes.
Redisanti‑crawlerspring-boot
0 likes · 6 min read


转转QAApr 1, 2022 · Information Security
This article presents a comprehensive design of the Cleaner anti‑crawler system, detailing its background, current challenges, related research, system architecture—including a Flink‑based data processing center, a strategy‑driven ban center, and a lightweight ban store—and evaluates its effectiveness in real‑time bot mitigation.
Backend Securityanti‑crawlerrate limiting
0 likes · 17 min read

The article presents a comprehensive design of the Cleaner anti‑crawler system, detailing its background, current challenges, related research, system architecture—including data processing, ban center, and ban store modules built on Flink, MQ, and Redis—implementation details, effectiveness evaluation, and concluding insights on achieving real‑time accuracy in protecting platform data.
BackendReal-timeRedis
0 likes · 17 min read

This article explains how to integrate the kk‑anti‑reptile anti‑crawler component into a Spring Boot application, covering system requirements, filter workflow, rule configuration, Maven setup, Redis and Apollo settings, as well as front‑end handling of the 509 response and captcha verification.
AxiosCAPTCHARedis
0 likes · 7 min read

This article introduces kk-anti-reptile, a Spring Boot‑based anti‑crawler filter that uses Redis, implements ip‑rule and ua‑rule, returns HTTP 509 with various captchas, and provides step‑by‑step Maven, configuration, and Axios front‑end integration instructions.
AxiosCAPTCHASpring Boot
0 likes · 6 min read

The article introduces kk-anti-reptile, a Spring Boot‑based anti‑crawler component that uses servlet filters, Redis, and configurable rule chains (IP and User‑Agent), explains its workflow, shows Maven and property configurations, and provides front‑end Axios interception code for handling 509 responses.
AxiosRedisSpring Boot
0 likes · 6 min read

This guide explains how the kk-anti-reptile component protects Spring‑Boot distributed systems from crawlers by using a servlet filter, configurable IP and User‑Agent rules, captcha challenges, Maven integration, and required Redis and Apollo settings, with full code examples.
CAPTCHAanti‑crawlerfilter
0 likes · 9 min read
