Mastering Dubbo’s Time Wheel: From Theory to Source Code

Hello, I am Su San. Today we dive into the time wheel, a practical mechanism often found in frameworks and interview questions.

Most people discuss the time wheel starting from Netty, but I will begin with Dubbo, where I first encountered it.

org.apache.dubbo.rpc.cluster.support.FailbackClusterInvoker

Failback is a cluster fault‑tolerance strategy that relies on the time wheel for scheduled retries.

When implementing scheduled tasks in Java, the ScheduledExecutorService is preferred over Timer. Two key methods are scheduleAtFixedRate (fixed‑rate based on start time) and scheduleWithFixedDelay (fixed‑delay based on end time). For Dubbo’s retry logic, scheduleWithFixedDelay is more appropriate.

Dubbo’s FailbackClusterInvoker uses a ConcurrentHashMap called failed to store failed requests and a HashedWheelTimer to schedule retries. The retry interval is defined by RETRY_FAILED_PERIOD, which defaults to 5 seconds.

The timer is created with a daemon thread named failback-cluster-timer, a tick duration of one second, a wheel size of 32, and a configurable maximum pending timeout count.

The wheel size must be a power of two, allowing fast index calculation using a bitmask ( mask = wheel.length - 1) and the expression idx = (int) (tick & mask).

During each tick, the worker thread waits for the next tick, processes cancelled tasks, transfers pending tasks to the appropriate bucket, and expires tasks whose deadline has arrived.

Cancelled tasks are stored in a LinkedBlockingQueue, while the original Netty implementation uses a lock‑free MpscLinkedQueue, a discrepancy noted in the source comments.

The core loop increments the tick counter after processing the current bucket, ensuring continuous operation until the timer is stopped.

Time Wheel Principle

The wheel is essentially an array where each slot represents a time slice (e.g., one second). By connecting the ends, the array forms a circular wheel, allowing tasks to be placed in slots based on delay % wheel.length. Tasks are first queued in a pending‑allocation queue and later moved onto the wheel at the appropriate time.

Tasks can also be cancelled, which removes them from the wheel before execution.

Time Wheel Source Code

The HashedWheelTimer constructor takes a thread factory, tick duration, time unit, number of ticks per wheel, and a maximum pending timeout count. The timer creates a daemon thread that runs the worker loop.

Key methods include newTimeout(task, delay, unit), which calculates the deadline as System.nanoTime() + unit.toNanos(delay) - startTime, and waitForNextTick(), which sleeps until the next tick, handling platform‑specific adjustments (e.g., Windows rounding to 10 ms).

When a tick occurs, the worker computes the bucket index using the mask, processes cancelled tasks, transfers pending tasks to buckets, and expires tasks whose deadline has passed.

Related Issues

Several GitHub issues discuss Dubbo’s time wheel, including memory‑leak concerns due to an unbounded failed map and the evolution of the implementation (e.g., issue #2425). Other discussions compare Netty’s use of HashedWheelTimer versus ScheduledThreadPoolExecutor and explore hierarchical time wheels in Kafka.