Interview Question: Handling MQ Consumption Bottlenecks – Diagnosis and Solutions with RocketMQ

During a second‑round interview at Ant Financial, the candidate was asked how to handle a bottleneck in MQ consumption, prompting a deeper discussion beyond the simple answer of horizontal scaling.

The author advises pausing to think, exploring the problem with the interviewer, and first identifying the root cause before suggesting optimizations.

Three discussion steps are recommended: determine how to detect a consumption bottleneck, investigate the root cause, and propose concrete solutions.

In RocketMQ, two primary indicators reveal a bottleneck: the number of delayed (backlogged) messages and the lastConsumeTime value; larger values for either suggest the consumer is struggling.

To locate the issue, check whether other consumer groups for the same topic also experience backlog, which often points to a client‑side problem. Searching the rocketmq_client.log for "flow" can reveal flow‑control logs, e.g., grep "flow" rocketmq_client.log, indicating that the consumer has been throttled due to backlog.

Further diagnosis involves tracing the consumer thread with jstack. Typical commands are:

ps -ef | grep java
jstack pid > j1.log

By examining multiple thread dumps, one can see if the ConsumeMessageThread_* remains in the same state, suggesting it is stuck, often on an external HTTP call; setting a timeout for such calls can alleviate the blockage.

Once the slow component is identified—commonly a third‑party service or database—targeted performance tuning is applied. If the backlog is acceptable (e.g., during traffic spikes like Double‑11) and TPS remains stable, horizontal scaling may be sufficient without deeper optimization.

The article concludes by questioning whether every backlog must be fixed, emphasizing that MQ’s purpose is to decouple and smooth traffic, and that occasional backlog can be intentional.