How RocketMQ Achieves Smart Push Consumption with Long Polling
This article explains RocketMQ's push and pull consumption modes, shows how its pseudo‑push implementation uses long polling to balance real‑time delivery and consumer pressure, and walks through the core source‑code mechanisms that hold and resume pull requests.
MQ Consumption Modes
Consumption modes describe how a consumer obtains messages from a message queue, typically categorized as push (the broker pushes messages to the consumer) or pull (the consumer actively pulls messages from the broker).
RocketMQ Implementation of Consumption Modes
RocketMQ, an open‑source high‑performance MQ from Alibaba, supports both push and pull. In most projects push is preferred because pull has higher latency and requires developers to manage consumption progress.
Why RocketMQ's Push Is Clever
RocketMQ’s push is actually a “pseudo‑push” built on pull. The consumer still initiates a pull request, but the broker holds the request until new messages arrive, giving the experience of an immediate push while allowing the consumer to control load.
Polling vs Long Polling
Polling repeatedly sends fixed‑interval requests regardless of data availability, leading to wasted requests and latency. Long polling holds the request on the server side until data becomes available, reducing unnecessary traffic and providing push‑like immediacy.
Push Consumption Source Code Exploration
The consumer runs a background thread that creates a PullRequest . It first checks whether pending messages exceed a threshold (by count or memory size). If pressure is high, it delays the next pull; otherwise it sends a pull request.
When a request reaches the broker and no messages are ready, the broker suspends the request via PullRequestHoldService.suspendPullRequest , storing it in a ManyPullRequest map.
When a new message arrives, the broker invokes NotifyMessageArrivingListener.arriving , which calls PullRequestHoldService.notifyMessageArriving to resume the held request and deliver the message to the consumer’s MessageListener .
Consumer Pressure Control
RocketMQ offers two pressure‑control strategies: one based on the number of unconsumed messages, the other on the memory size of those messages. If either exceeds configured limits, the consumer postpones further pulls.
Server Holds Pull Request
If no messages are found, PullRequestHoldService classifies and stores the request in a ManyPullRequest structure backed by a ConcurrentHashMap .
Server Notifies Consumer
Upon message arrival, the broker’s NotifyMessageArrivingListener triggers the hold service to resume the request, allowing the consumer to receive the new message immediately.
Conclusion
RocketMQ balances real‑time delivery and consumer load by implementing push via long polling, a pattern also used in other systems such as Nacos. Understanding this design makes it easier to grasp similar mechanisms in other frameworks.
Sanyou's Java Diary
Passionate about technology, though not great at solving problems; eager to share, never tire of learning!
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.