How Tencent Cloud’s Virtual Queue Enables Seamless Compatibility for RocketMQ 5.x Remoting Clients

Background and New Features in RocketMQ 5.x

Apache RocketMQ 5.x introduces two major updates: (1) decoupling of storage and compute to improve scalability and cloud‑native adaptability, and (2) a POP consumption mode that moves load‑balancing logic from the client to the broker.

To leverage these features, the community released a lightweight gRPC‑based client. Existing users still using the Remoting protocol cannot benefit from the new capabilities without upgrading their SDK.

Challenges for Existing Remoting Clients

In the 4.x architecture, clients maintain direct long‑lived connections to every broker. RocketMQ 5.x switches to a proxy‑forwarding model, requiring the Remoting SDK to be upgraded to at least v4.9.5, which adds a BrokerName field.

Upgrading is difficult for many enterprises: it demands extensive testing, and over 90% of users still run versions below v4.9.5, making the migration costly and risky.

Tencent Cloud Virtual‑Queue Compatibility Solution

To address these limitations, Tencent Cloud Message Queue for RocketMQ offers a virtual‑queue scheme that fully supports legacy Remoting clients without code changes.

Server‑side compatibility layer that abstracts the underlying queues, removing the SDK version dependency.

Transparent migration of POP consumption logic to the server, enabling message‑level load balancing for Remoting clients.

Virtual Queue Definition

A topic consists of multiple queues distributed across brokers. The virtual queue hides the real queues from the client, presenting a virtual broker (vBroker) with a different set of queues. The mapping between virtual and real queues is managed by the server, so the client no longer needs to specify a target broker.

Transparent Consumption‑Mode Conversion

The server converts the Remoting client’s PULL requests into POP requests on the real queues, handling queue selection and dynamic adjustments to balance load while keeping latency low.

Interaction steps:

Client sends a PULL request to the virtual queue; the server translates it to a POP request on the real queue.

After processing, the client submits its consumption offset, which the server converts into a POP receipt handle with selective acknowledgment and automatic invisible‑time renewal.

This conversion is performed in the Proxy layer and is completely transparent to the client and the storage brokers, requiring no persistent state and having no impact on Proxy scaling.

Eliminating Client‑Side Rebalance

When consumer groups change, traditional Remoting clients trigger a rebalance that can destabilize large clusters. With virtual queues, the server masks changes in real‑queue counts and consumer lists, removing the need for client‑side rebalance entirely and greatly improving system stability.

Maintaining Queue Order Consumption

Order‑sensitive scenarios are supported by:

Server‑side assignment of each queue to a single online client, ensuring exclusive pulling.

Client‑side sequential delivery of messages to the business logic.

Conclusion

The article outlines RocketMQ 5.x’s architectural evolution and the two main compatibility challenges for Remoting clients: SDK version dependence and consumer‑queue binding limitations. Tencent Cloud’s virtual‑queue solution resolves these by abstracting queues and transparently converting consumption modes, enabling legacy clients to seamlessly adopt the new architecture, reduce migration risk, and achieve stable, ordered, and load‑balanced message consumption at scale.