kk-anti-reptile: Spring Boot Anti‑Crawler Component and Integration Guide

When building e‑commerce sites, competitors often use crawlers to scrape data; without protection the site can be overwhelmed. The article presents kk-anti-reptile , a Spring Boot‑compatible anti‑crawler component designed for distributed systems.

System requirements : Spring Boot 1.x or 2.x and a Redis instance.

Workflow : The component registers a Servlet Filter via FilterRegistrationBean . Inside the filter a responsibility‑chain pattern applies configurable rules, currently ip‑rule (limits request count per time window) and ua‑rule (filters based on User‑Agent).

If a request violates a rule, the filter blocks it, returns HTTP status 509 , and serves a captcha page. Captchas are offered in six formats (Chinese characters, alphanumeric, arithmetic; each with static image or GIF), making automated solving extremely difficult.

Integration : Add the Maven dependency:

<dependency>
  <groupId>cn.keking.project</groupId>
  <artifactId>kk-anti-reptile</artifactId>
  <version>1.0.0‑SNAPSHOT</version>
</dependency>

Enable the component with the property:

anti.reptile.manager.enabled=true

On the front end, intercept a 509 response with Axios, open a verification window, render the returned HTML, and inject the baseUrl parameter:

import axios from 'axios';
import { baseUrl } from './config';

axios.interceptors.response.use(
  data => data,
  error => {
    if (error.response.status === 509) {
      let html = error.response.data;
      let verifyWindow = window.open('', '_blank', 'height=400,width=560');
      verifyWindow.document.write(html);
      verifyWindow.document.getElementById('baseUrl').value = baseUrl;
    }
    return Promise.reject(error);
  }
);
export default axios;

Additional notes : Apollo client users must enable bootstrap and set apollo.bootstrap.enabled=true . If Redisson is used, the component automatically obtains a RedissonClient ; otherwise, provide Redis connection settings such as spring.redisson.address=redis://192.168.1.204:6379 and password.