How JD’s “Woodpecker” System Ensures Real‑Time E‑Commerce Activity Monitoring

Background

Operational activities are a key post‑sale promotion method on e‑commerce platforms; high‑quality campaigns attract users and improve conversion rates. Technical teams focus on non‑content quality such as accessibility, performance metrics, and cross‑platform compatibility.

JD’s online activity pages span App, PC, M‑site, and WeChat, with countless daily activities—especially during major sales like 618 and Double‑11, reaching tens of thousands of events. Ensuring these activities are accessible and error‑free is critical for user experience.

System Overview

The Woodpecker system provides fast, accurate, and comprehensive online activity monitoring across all platforms, offering multiple daily full‑coverage scans and 10‑minute interval checks for key homepage pages, enabling immediate alerts before users encounter problems.

Functional Directions

Activity expiration issues

404 link problems

Access timeout

Empty page content

Holiday mismatch

Missing HTTPS

Platform adaptation errors

Plus‑member pricing checks

Performance Directions

Excessive first‑screen load time

Oversized images

Heavy first‑screen traffic

No caching for static resources

Short cache duration

Invalid cache

Static resources using cookies

Uncompressed data transmission

Non‑reused HTTP connections

Frequent illegal status codes (302, 404)

Invalid resource requests

Unoptimized jQuery usage

Architecture

The system architecture is simple, built on Spring‑based backend services.

Activity List Creation

Real‑time activity lists are obtained via APIs, with static configurations for fixed‑location activities.

Monitoring System

The scheduling module reads the activity list, creates an Activity model for each entry (including parent hierarchy), and pushes them into a thread pool.

Each monitoring thread launches an external Node.js process to fetch page content, passes it to the checking module, and either alerts issues or retrieves child activities for further checks.

Monitoring Thread

Threads use a headless Chrome browser via Puppeteer to obtain rendered DOM, enabling fast, UI‑less page access.

Puppeteer, a Node.js framework from Google, supports full browser control, multi‑process concurrency, and can handle login‑required pages, app‑specific pages, and request/response mocking.

Alert Module

The alert module notifies responsible personnel and records issues in a backend database, with configurable recipients for each monitoring type.

Fix Confirmation System

After alerts, a confirmation system tracks issue resolution progress, identifying long‑standing problems for further reporting.

Conclusion

The Woodpecker system scans tens of thousands of activities daily at a throughput of under 100 items per minute, delivering rapid detection and resolution to improve user experience and drive business benefits.