Seamlessly Migrate from RocketMQ to Apache Pulsar with the RoP Plugin

Overview

RoP (RocketMQ Protocol) is a pluggable protocol‑handler for Apache Pulsar that implements the native RocketMQ wire protocol. By loading the RoP plugin on a Pulsar broker, RocketMQ clients can connect to Pulsar without any code changes.

Motivation

Both RocketMQ and Pulsar have large user bases. Migrating applications from RocketMQ to Pulsar traditionally requires rewriting the protocol layer, which is costly. RoP provides a drop‑in compatibility layer so that existing RocketMQ producers and consumers can use Pulsar’s features—compute‑storage separation, multi‑tenant support, cross‑region replication, tiered storage, and Pulsar Functions—without modification.

Architecture

RoP is implemented as a Protocol Handler (PIP‑41) that exposes the Netty channel and the Pulsar BrokerService instance to the plugin. The handler intercepts client requests, translates them to Pulsar API calls, and forwards the responses back to the RocketMQ client.

Protocol Mapping

Topic Lookup – Pulsar performs a lookup request to locate the owner broker; RocketMQ sends GET_ROUTEINTO_BY_TOPIC to obtain routing information before establishing a TCP connection.

Produce – RocketMQ produce requests are mapped to Pulsar’s PersistentTopic append operations, which ultimately write to Apache BookKeeper ledgers.

Consume – Consume requests are translated to Pulsar consumer reads from the same BookKeeper ledger.

Offset / MessageID – RocketMQ uses a numeric offset. Pulsar uses a MessageId composed of ledgerId, entryId, and partitionId. RoP provides a deterministic mapping between offset and MessageId.

Storage Integration

Pulsar’s storage layer is built on Apache BookKeeper. RoP maps RocketMQ’s commitLog and queueLog operations to BookKeeper ledger management, allowing seamless persistence of messages.

Message Conversion

Headers and payloads are preserved during translation. An additional 8‑byte field is added to distinguish tag‑type messages.

Release and Usage

RoP version 0.1.0 is publicly available. The plugin can be run as a standalone service or deployed inside an existing Pulsar cluster.

Download the plugin and documentation:

https://github.com/streamnative/rop/blob/master/README.md

Example applications (produce/consume) are located at:

https://github.com/streamnative/rop/tree/master/examples/src/main/java/org/streamnative/rocketmq/example

Source code and issue tracker:

https://github.com/streamnative/rop