How TDMQ Pulsar Enables Cross‑Region Replication for Global Data Archiving and Disaster Recovery

Background

Since 2024‑09‑06 TDMQ Pulsar professional clusters support two‑level cross‑region replication: metadata‑level for disaster‑recovery and message‑level for global data archiving.

Replication Types

Metadata‑level replication synchronizes namespaces, topics, subscriptions, roles, etc., enabling business continuity when an entire region fails.

Message‑level replication copies message payloads, ordering, tags and types, allowing unified archiving across regions.

Metadata‑level Replication Workflow

Establish VPC connectivity between the two regions (e.g., via Cloud Connect).

Synchronize cluster metadata (namespaces, topics, subscriptions, roles) from source to destination.

When the source region becomes unavailable, update the service domain’s DNS to point to the disaster‑recovery region so clients keep the same endpoint.

After the source region recovers, optionally replay messages generated in the DR region back to the primary region and switch DNS back.

Message‑level Replication Workflow (example: Shanghai → Beijing)

Producer1 publishes to Topic1 on the Shanghai cluster.

The Shanghai cluster persists the message to BookKeeper.

BookKeeper acknowledges the write and the Replicator cursor reads the message.

The Replicator cursor uses an internal producer (Producer‑R) to forward the message to the Beijing cluster’s Topic1.

Beijing cluster writes the message to its BookKeeper and sends an ACK back to the Shanghai cursor, which then proceeds with the next message.

Configuration Steps

Create a professional cluster in the target region and enable the “cross‑region replication” switch, selecting the source cluster as the replication source.

In the console, create a replication link:

Specify a link name.

Choose the replication type (metadata‑level or message‑level).

Select source and target clusters (only clusters with the same cluster ID are listed).

Define the replication scope: cluster‑level, namespace‑level, or topic‑level.

After creation, monitor the replication task on the monitoring page for actual throughput and backlog.

Performance Example (Financial‑industry use case)

A financial platform (FiT) migrated its messaging layer to TDMQ Pulsar and enabled cross‑region replication. After migration, end‑to‑end latency decreased by ~80 % and message backlog reduced by ~70 %, demonstrating high‑throughput, low‑latency characteristics.