Java Sogou Image Crawler: Fetching and Downloading Images with WebMagic

This tutorial explains how to use Java and the WebMagic framework to crawl thousands of images from Sogou image search, parse the JSON responses to extract image URLs, and download the pictures locally using multithreaded processing and custom HTTP utilities.

HTTPImage DownloadWebMagic

0 likes · 17 min read

Java Sogou Image Crawler: Fetching and Downloading Images with WebMagic

Programmer DD

Jul 31, 2021 · Backend Development

Build a Spring Boot Web Crawler with WebMagic, MyBatis, and MySQL

This tutorial demonstrates how to combine Spring Boot, WebMagic, and MyBatis to crawl Zhihu pages, configure Maven dependencies, set up MySQL data sources, define entity and mapper classes, and schedule the crawler to run periodically, providing a complete Java web‑crawling scaffold.

JavaMyBatisScheduler

0 likes · 14 min read

Build a Spring Boot Web Crawler with WebMagic, MyBatis, and MySQL