How Dubbo Leverages Hashed Wheel Timer for Efficient Retries and Heartbeats

Time Wheel Overview

The time wheel is essentially an array that loops back on itself, where each slot represents a time unit (e.g., one second). Tasks are placed into slots based on their delay modulo the wheel size, and a worker thread advances the wheel at fixed tick intervals.

Dubbo Usage

Dubbo adopts Netty's HashedWheelTimer for several purposes, including failback retry, request timeout detection, and heartbeat sending. The FailbackClusterInvoker class uses scheduleWithFixedDelay to schedule retry tasks, with RETRY_FAILED_PERIOD defining the retry interval (default 5 seconds).

Failed requests are stored in a ConcurrentHashMap called failed. The retryFailed method iterates over this map, re-invoking each request; successful calls remove the entry, while failures are logged and rescheduled.

Heartbeat tasks are handled by HeartbeatTimerTask, which sends a heartbeat packet and then re‑queues itself in the timer.

Hashed Wheel Timer Mechanics

The timer is created with a thread factory, tick duration, time unit, ticks per wheel, and a maximum pending timeout count. The wheel size is always a power of two, enabling fast index calculation using mask = wheel.length - 1 and bitwise & operations.

When a new timeout is scheduled, the deadline is computed as System.nanoTime() + unit.toNanos(delay) - startTime. The worker thread repeatedly calls waitForNextTick(), processes cancelled tasks, transfers pending tasks to appropriate buckets, expires tasks in the current bucket, and increments the tick counter.

Cancellation is handled via HashedWheelTimeout#cancel, which adds the timeout to a cancelledTimeouts queue for later cleanup.

Configuration Details

Thread name: failback-cluster-timer Tick interval: 1 second

Wheel size: 32 (configurable, default 512)

Max pending timeouts: configurable (default -1 for unlimited)

Related Issues

Several GitHub issues discuss the evolution of Dubbo's time wheel implementation, memory‑leak concerns with the unbounded failed map, and comparisons with Netty's usage of HashedWheelTimer versus ScheduledThreadPoolExecutor.