This tutorial demonstrates how to use Alibaba's Canal in cluster mode with Zookeeper to capture MySQL row‑based binlog events, publish them via RocketMQ, and consume them with a custom listener that transforms inserts, updates, and deletes into synchronized documents in an Elasticsearch index.
This step‑by‑step tutorial shows how to configure Alibaba's Canal to capture MySQL binlog changes, route them through RocketMQ, and index the data into Elasticsearch, covering cluster mode, MySQL and Elasticsearch setup, RocketMQ topic creation, Canal properties, and consumer implementation.
This article provides a detailed overview of popular message queue systems, explains their core concepts such as decoupling and eventual consistency, and compares the advantages and disadvantages of Kafka, RocketMQ, RabbitMQ, and other notable MQ solutions for high‑concurrency scenarios.
This guide demonstrates how to integrate RocketMQ 4.8.0 into a Spring Boot 2.4.12 application, covering dependency setup, configuration, sending and receiving normal, ordered, cluster/broadcast, and transactional messages, complete with code snippets and execution results to help developers implement reliable messaging patterns.
This article explains the three core benefits of message queues, compares major MQ products, details RocketMQ's advantages, internal models, reliability mechanisms, load‑balancing, storage design, and practical code examples, guiding engineers to make informed architectural decisions.
This article explains RocketMQ's advanced features—transactional messaging, ordered messaging, SQL‑based filtering, delayed messaging, and global high‑availability—detailing their principles, implementation steps, practical demos, and how they solve consistency challenges in large‑scale e‑commerce systems.
This article explains RocketMQ's master‑slave replication mechanism, comparing synchronous and asynchronous modes, detailing metadata and message data copying processes, and showing how synchronous guarantees are implemented using CompletableFuture to ensure high availability.
This guide explains how to achieve reliable message consistency in a Spring Boot 2.3.9 application using RocketMQ 4.8.0 by outlining the transactional message flow, broker handling, project structure, code implementation, and testing procedures.
This article explains RocketMQ transaction messages by covering their use cases, underlying mechanisms, and a step‑by‑step implementation that demonstrates how to ensure eventual consistency between message production and local database transactions in distributed Java back‑end systems.
This article provides a comprehensive walkthrough of RocketMQ 4.9.x consumption architecture, covering the four core roles, publish‑subscribe model, storage structures, load‑balancing, long‑polling, concurrent and ordered consumption flows, progress persistence, and retry mechanisms, with illustrative diagrams and code snippets.
This article explains the ACID properties of transactions and compares various implementation approaches—including database‑level mechanisms, Spring single‑datasource transactions, and distributed transaction frameworks such as Seata and RocketMQ—while analyzing their advantages, drawbacks, and suitable scenarios.
After a massive message backlog in RocketMQ, the author discovered that identical clientIds generated by Docker containers using host network mode caused broker load‑balancing errors, leading to slow consumption; the article explains the root cause, code analysis, and how setting a unique clientId resolves the issue.
This article traces the development of message queues from early decoupling solutions like ActiveMQ and RabbitMQ, through high‑throughput designs such as Kafka and RocketMQ, to modern platform‑centric systems like Pulsar, while detailing core concepts, architecture diagrams, storage mechanisms and trade‑offs.
This article examines the false empty response issue caused by long‑polling coupling in RocketMQ 5.0 SimpleConsumer, analyzes existing approaches, and presents an adaptive, stateless proxy solution that decouples client and broker polls, adds notification, and includes metrics and configuration guidance.
The article traces two decades of message‑queue evolution—from early decoupling tools like ActiveMQ, through Kafka’s high‑throughput log model, to Pulsar’s layered, cloud‑native architecture—explaining core concepts, storage designs, and trade‑offs that guide choosing the right MQ for modern workloads.
This article explores RocketMQ's high availability mechanism, focusing on DLedger's implementation of Raft-based leader election for distributed message middleware, covering both traditional and Controller mode architectures.
This guide explains why and how to read open‑source Java code, covering essential JDK knowledge, design patterns, official documentation, module structure, demo‑first approaches, purposeful reading strategies, class‑name clues, class hierarchy analysis, commenting practices, and leveraging related resources.
This article provides a comprehensive overview of RocketMQ, covering its origin, core concepts, component roles, cluster deployment architecture, workflow, feature details, persistence mechanisms, and cleanup policies, offering developers a solid foundation for using this high‑performance messaging middleware.
This article examines RocketMQ's high‑availability evolution—from early master‑slave and Raft‑based designs to the v5 DLedger fusion model—detailing replica groups, data sharding, election mechanisms, replication strategies, metric trade‑offs, log‑divergence handling, controller roles, heartbeat optimizations, and comparisons with Kafka and Pulsar, all illustrated with diagrams and code snippets.
This article walks through RocketMQ 4.9.x’s consumption architecture, explaining the roles of NameServer, Broker, Producer and Consumer, the publish‑subscribe model, storage structures, load‑balancing algorithms, long‑polling, concurrent and ordered consumption, progress persistence, and the built‑in retry mechanism.
This article explains the architecture of a payment center that unifies internal payment services and external third‑party integrations, details the three‑step payment flow, discusses common issues such as order timeout and result reliability, and demonstrates how template‑method and strategy patterns together with RocketMQ delay queues solve these problems in a SpringBoot backend.
The article explains when adding consumers can reduce RocketMQ backlog, the reasons for delayed message pulling, handling slow external services, and details six different MessageQueue allocation strategies—including average, round‑robin, custom, machine‑room, nearby, and consistent‑hash—accompanied by Java code examples.
This article explains why ordered message processing is crucial for business workflows, outlines how to maintain order during production, storage, and consumption in RocketMQ, and describes the locking mechanisms required to achieve reliable sequential consumption.
RocketMQ 5.1.0 introduces a tiered storage module that offloads messages from local disks to cheaper storage, enabling longer retention without impacting hot data performance, and the article explains its design, implementation details, configuration steps, and current challenges.
The article explains how RocketMQ 5.1.0 introduces a tiered storage module that offloads messages to cheaper media, describes its design, architecture layers, quick‑start configuration, upload and read mechanisms, prefetch cache, fault recovery, current development plans, and remaining challenges.
This article provides a comprehensive overview of major message‑queue middleware—including RocketMQ, Kafka, Pulsar, and RabbitMQ—covering fundamental concepts such as tags, groups, offsets, architectural components, storage mechanisms, transaction workflows, rebalance strategies, and recent developments, while comparing their features and performance characteristics.
This article provides a detailed side‑by‑side comparison of five popular message‑queue systems—Kafka, RabbitMQ, ZeroMQ, RocketMQ, and ActiveMQ—covering documentation, programming languages, supported protocols, storage, transactions, load balancing, clustering, management UI, availability, duplication handling, throughput, subscription models, ordering, acknowledgments, replay, retry, concurrency, and includes promotional information from the author.
This article breaks down RocketMQ’s storage architecture, explaining how its commitlog, consumequeue, and index files work together to ensure reliable persistence, quick offset-based lookups, and efficient message consumption in a publish‑subscribe model.
This article explores a systematic approach to becoming a better programmer, covering learning mindset, selecting role models, applying knowledge, coding principles, performance considerations, and refactoring techniques, illustrated with a RocketMQ file‑prewarm case study and practical guidelines for writing maintainable, high‑quality code.
This article explains how Apache RocketMQ 5.0 integrates standardized distributed tracing via OpenTelemetry, detailing the underlying span model, semantic conventions for messaging, automatic and manual instrumentation options, configuration steps, a complete example workflow, and how to export traces to Alibaba Cloud SLS and ARMS for observability.
This article examines the various reasons why RocketMQ can deliver the same message to consumers multiple times, covering send‑retry mechanisms, consumer‑side exceptions, offset commit and persistence failures, master‑slave synchronization issues, rebalance events, and long‑processing cleanup, and offers guidance on designing idempotent solutions.
This tutorial walks through practical implementations of RocketMQ transaction, ordered, filtered, and delayed messages using native Java code and Spring Boot, explaining the underlying concepts, showing complete code examples, and highlighting how to configure producers, consumers, and listeners for reliable distributed messaging.
This article explains how RocketMQ creates and manages threads, covering single‑thread creation via Runnable and Thread inheritance, the ServiceThread abstract class, ThreadPoolExecutor configuration, custom thread factories, and the importance of descriptive thread names for debugging.
This guide explains why reading open‑source code matters—from interview prep to design mastery—and offers 18 practical steps, including mastering the JDK, understanding design patterns, using official docs, exploring module structures, starting with demos, and adopting purposeful, efficient reading habits.
A detailed investigation reveals that RocketMQ can report successful writes to the broker while consumers see minutes‑long delays because the transient store pool and commit‑log forwarding mechanisms delay the transfer of messages from the broker’s memory to the consumer queue.
This article presents a detailed case study of RocketMQ's fast‑failure mechanism, explains the root causes of a massive message‑write surge, analyzes broker thread models, and proposes topic‑level rate‑limiting solutions with concrete implementation and code examples.
This guide explains why reading source code matters and presents 18 practical techniques—covering JDK fundamentals, design patterns, official documentation, module analysis, demo‑first approach, purposeful reading, and effective note‑taking—to help developers confidently explore Java open‑source projects like RocketMQ.
This article explains RocketMQ's consumer startup process in pull mode, detailing the underlying push‑like implementation, configuration validation, instance naming, client initialization, load‑balancing, wrapper handling, offset storage, scheduled tasks, and trace dispatching, accompanied by Java code examples.
This article describes how a review middle‑platform for a video service standardizes communication protocols with RocketMQ, defines unified data processing flows, and implements an automated business‑access detector with Java code to reduce duplication and improve development efficiency.
Facing a four‑hour coupon‑calculation bottleneck at eLong, I explored Storm’s stream‑processing model, identified the data‑extraction stage as the weak link, refactored the hotel‑pull service, leveraged RocketMQ for threading, and ultimately reduced full‑batch processing time to just over an hour.
The article uses a safety‑principle analogy to introduce three real‑world backend incidents—a Java ternary‑operator NullPointerException caused by unintended unboxing, a RocketMQ message‑storm triggered by swapped parameters, and a network‑path latency spike—showing how language semantics, messaging contracts, and infrastructure topology each can create production bugs.
This guide explains how to use Alibaba Cloud EventBridge to create and manage RocketMQ 4.0, 5.0, and self‑hosted Apache RocketMQ instances, covering architecture, API parameters, VPC and security settings, and step‑by‑step procedures for event‑driven integration and message routing.
vivo’s Internet Storage team details how it operates RocketMQ for low‑latency online services and Kafka for massive big‑data pipelines, outlines resource isolation, traffic balancing, intelligent throttling, and governance practices, and describes its migration from RabbitMQ and planned shift from Kafka to cloud‑native Pulsar.
This article walks through every stage of a RocketMQ message—how producers create and route messages, the storage mechanisms and zero‑copy techniques used by brokers, high‑availability modes, consumption models, ordering guarantees, and the automatic cleanup policies that finally retire messages.
This article explains the concept of idempotency, illustrates why repeated operations must yield identical results, and details how RocketMQ’s components—NameServer, Broker, Producer, and Consumer—can be designed with unique message IDs and acknowledgment mechanisms to achieve reliable, duplicate‑free messaging in backend systems.
Apache RocketMQ 5.0 introduces a stateless proxy, gRPC‑based multi‑language SDK with immutable APIs, enhanced error handling, SimpleConsumer, and pop consumption, and the article outlines the technical improvements, roadmap, and step‑by‑step guide for developers to set up the environment, understand the new protocols, and contribute language client implementations.
This article introduces the fundamentals of message middleware, outlines its core components, and compares RocketMQ and Kafka architectures to show how they achieve high‑availability through replication, leader election, and coordinated producer‑consumer interactions.
This tutorial explains why message queues are essential, compares popular MQ products, details RocketMQ's core components, and provides step‑by‑step instructions—including environment setup, broker configuration, firewall rules, and monitoring—to deploy a fault‑tolerant RocketMQ cluster on CentOS 7.
This article explains RocketMQ's load balancing process, including when rebalancing occurs, the impact on message consumption, common strategies such as average allocation and consistent hashing, and practical recommendations like minimizing client churn and choosing appropriate strategies for cloud‑native deployments.
This article explains how RocketMQ's message lifecycle can be fully observed using OpenTelemetry‑based metrics, covering producer, broker, and consumer stages, and shows practical monitoring, alerting, and troubleshooting practices for cloud‑native deployments.
This article provides an in-depth comparison of RabbitMQ, RocketMQ, and Kafka, detailing their core components, exchange types, message durability, acknowledgment mechanisms, TTL, dead‑letter queues, load balancing, ordering, transaction handling, high‑availability configurations, and practical solutions for common messaging challenges.
This article provides an in‑depth tutorial on RocketMQ, covering normal, ordered, delayed, transactional, and batch messages, their production and consumption patterns, retry mechanisms, message filtering, and dead‑letter handling, complete with Java code examples for each scenario.
This article explains RocketMQ's broker storage architecture, detailing the roles of CommitLog, ConsumeQueue, and IndexFile, and describes how messages are formatted, indexed, and written to disk using memory‑mapped files to achieve high throughput and reliability.
This article explains the concept of distributed transactions, presents typical microservice scenarios that generate them, and demonstrates a complete implementation using RocketMQ transactional messages and SEATA, including database schema, producer and consumer code, transaction listeners, and idempotent consumption handling.
This article explains RocketMQ's load‑balancing mechanisms, when rebalancing occurs, how different strategies like average allocation and consistent hashing affect consumer behavior, and provides practical recommendations to minimize message backlog and consumption spikes in cloud‑native environments.
This article explains how RocketMQ integrates with Prometheus and Grafana to provide comprehensive metrics, tracing, and logging, detailing the exporter architecture, deployment choices, span topology, dashboard examples, and ARMS‑based alerting for cloud‑native message‑queue observability.
This article explains the background of RocketMQ clusters, demonstrates typical Java consumer and producer usage, and introduces the AOP‑based HunterConsumer and HunterProducer components that simplify lifecycle management, configuration, and code reuse for backend developers working with messaging systems.
This article provides a comprehensive overview of RocketMQ messaging concepts—including synchronous, asynchronous and one‑way sending, ordered and delayed messages, transaction handling, batch processing, tag/SQL filtering, retry mechanisms, and dead‑letter queue management—accompanied by Java code examples for each scenario.
This article explains the concept of distributed transactions, presents typical micro‑service scenarios that generate them, and demonstrates a complete solution using RocketMQ transaction messages together with SQL table definitions and Java code for producers, transaction listeners, and consumers to achieve eventual consistency.
This article explains RocketMQ’s core components, the rocketmq‑remoting module, its Netty‑based protocol design, message structure, communication flow, and the Reactor multi‑thread model that together ensure high‑performance, reliable messaging in distributed backend systems.
RocketMQ 5.0 introduces a brand‑new unified API built on gRPC, refines client types, adds four message categories, pre‑fetches topic routes, embeds a telemetry channel, and provides OpenTelemetry‑based tracing and metrics, all while offering a cloud‑native SDK repository for developers.
This article details the challenges of scaling RabbitMQ, the evaluation of RocketMQ versus Pulsar, the design of a new messaging middleware platform with high availability, performance, and rich features, and the step‑by‑step migration strategy that enabled seamless, low‑cost transition for massive business traffic.
This article explains how RocketMQ implements distributed transaction messages using a two‑phase commit model, detailing the problems with RPC and ordinary MQ, the step‑by‑step workflow, relevant code snippets, and how consistency is ensured across microservice subsystems.
This article explains how RocketMQ implements scheduled and delayed messages using fixed delay levels, the internal storage flow, code examples for setting delay levels, and advanced techniques such as RocksDB integration to achieve arbitrary time precision for delayed delivery.
This article explains the challenges of point‑to‑point data sync, introduces RocketMQ Connect as a cloud‑native solution that decouples upstream and downstream, details its architecture, connectors, REST API, metrics, deployment modes, and provides a step‑by‑step guide to building custom connectors for use cases such as CDC, data lakes, and system migration.
This article provides a detailed tutorial on installing and configuring RocketMQ on a Mac M1 using Docker, covering source‑code and Docker deployments, container setup, broker configuration, dashboard launch, message testing, and common troubleshooting tips.
This article details how to set up Apache RocketMQ on a Mac M1 using Docker, covering source‑code deployment, Docker container configuration, broker and nameserver setup, dashboard launch, message testing, and troubleshooting common connection issues.
This article explains the design and implementation steps for a multi‑active (geo‑distributed) system, covering data‑center segmentation, traffic scheduling, RPC routing, database and Redis sharding, distributed IDs, message queue handling, job coordination, and cut‑over procedures.
This article examines a puzzling 'prevent gc' comment in RocketMQ source, explains how inserting Thread.sleep(0) creates a JVM safepoint to avoid long GC pauses, explores counted vs uncounted loops, and demonstrates practical code modifications to improve performance.
This article explains how RocketMQ creates and manages threads, covering single‑thread techniques, the ServiceThread abstract class, ThreadPoolExecutor fundamentals, custom thread‑pool wrappers, and the critical role of descriptive thread names for debugging and performance monitoring.
This article investigates the mysterious Thread.sleep(0) call found in RocketMQ source, explaining how it interacts with JVM safepoints, why it appears in counted loops, and how changing loop indices from int to long can avoid long stop‑the‑world pauses, backed by code examples and references.
The article walks through an interview scenario on RocketMQ, explaining when increasing consumer instances speeds up message consumption, the conditions that cause consumer pull delays, and the six built‑in queue‑allocation strategies with code examples and practical trade‑offs.
This article examines the puzzling 'prevent gc' comment and Thread.sleep(0) call in RocketMQ’s source, explains how it leverages JVM safepoint mechanics to trigger garbage collection, discusses counted vs uncounted loops, and demonstrates practical code modifications to improve performance.
This comprehensive guide explains the core concepts, deployment modes, evaluation criteria, and detailed pros and cons of the four major message‑queue platforms—Kafka, RabbitMQ, RocketMQ, and ActiveMQ—helping engineers make informed decisions for interview preparation and system design.
This article traces RocketMQ's transformation from a traditional message queue to a cloud‑native, ultra‑fusion platform that integrates messaging, event‑driven architecture, and streaming, highlighting its historical stages, architectural upgrades, multi‑language SDKs, stateless consumption, and future cloud‑native trends.
This article explains the role of RocketMQ's NameServer, detailing its registration process, data structures, periodic heartbeat checks, failure cleanup, and how producers and consumers retrieve routing information to send and receive messages efficiently.
This article explains why delayed queues are needed for scenarios like unpaid orders or auto‑generated comments, compares built‑in and third‑party solutions, and provides a detailed design and implementation guide for a Redis‑based delayed‑queue service, including version‑1.0 features, version‑2.0 optimizations, and multi‑node deployment.
The article explains fundamental concepts, common use cases, and selection criteria for four popular message‑queue platforms—Kafka, RabbitMQ, Pulsar, and RocketMQ—detailing each system’s architecture, strengths, and drawbacks so readers can match workload, scalability, ordering, latency, and operational needs to the most suitable middleware.
To address RabbitMQ’s scalability, reliability, and feature limitations, Vivo’s middleware team evaluated RocketMQ and Pulsar, selected RocketMQ, and built a next‑generation message middleware platform with an AMQP‑proxy gateway, metadata services, and high‑availability mechanisms, enabling seamless, high‑throughput migration and richer messaging capabilities.
This article explains RocketMQ's push and pull consumption modes, shows how its pseudo‑push implementation uses long polling to balance real‑time delivery and consumer pressure, and walks through the core source‑code mechanisms that hold and resume pull requests.
This article explains how RocketMQ reduces message loss and duplicate consumption through synchronous sending, retry mechanisms, multi‑master brokers, synchronous disk flushing, master‑slave replication, at‑least‑once delivery, and idempotent processing techniques.
This article introduces the core concepts, components, exchange types, reliability mechanisms, ordering, delay, transaction, high‑availability, and load‑balancing strategies of three popular message‑queue systems—RabbitMQ, RocketMQ, and Kafka—while also discussing common challenges such as message ordering, delayed consumption, reliability, idempotence, and backlog handling.
This article explains the challenges of maintaining atomicity across multiple downstream services in distributed systems and details how RocketMQ's transactional message feature provides a two‑phase commit mechanism, lifecycle, implementation details, and practical usage to achieve eventual consistency.
This article provides a comprehensive overview of the core components, exchange types, TTL, confirmation mechanisms, dead‑letter queues, and reliability features of RabbitMQ, RocketMQ, and Kafka, along with practical guidance on ordering, delayed consumption, transaction handling, high availability, load balancing, and message deduplication in distributed messaging systems.
This article surveys common delayed‑message implementations—including external storage, RocksDB, Redis, and open‑source MQs like RocketMQ, Pulsar, and QMQ—detailing their architectures, advantages, disadvantages, and practical considerations for distributed asynchronous messaging systems.
This article provides a comprehensive overview of modern message‑queue systems, detailing RabbitMQ components and exchange types, RocketMQ’s core architecture and high‑availability features, and Kafka’s fundamental concepts, controller mechanisms, and consumer rebalance strategies, while also covering reliability, ordering, and dead‑letter handling techniques.
This article explains the fundamental components and mechanisms of RabbitMQ, RocketMQ, and Kafka—including exchanges, TTL, confirm and return listeners, consumer ACK/NACK, dead‑letter queues, delay and ordered messages, transactional flows, high‑availability setups, load balancing, partitioning, leader election, offset handling, and practical solutions to common messaging challenges.
This article surveys common delayed‑message solutions in distributed asynchronous messaging, evaluates implementations based on external storage, databases, RocksDB, Redis, and open‑source MQs like RocketMQ, Pulsar and QMQ, and discusses their advantages, drawbacks, and practical design considerations.
This article provides an in‑depth technical guide to three major message‑queue systems—RabbitMQ, RocketMQ, and Kafka—covering their core components, exchange and routing mechanisms, durability features, consumer acknowledgment, dead‑letter handling, load balancing, and practical strategies for reliability, ordering, and scaling in distributed backend applications.
Facing scaling, high‑availability, and feature limitations of RabbitMQ, the vivo middleware team evaluated RocketMQ and Pulsar, chose RocketMQ, and detailed a seamless migration strategy—including a message gateway, metadata mapping, high‑performance push, consumption controls, and operational benefits that boosted TPS and reduced resource usage.
This article analyzes the challenges of distributed transactions in multi‑node systems and compares various solutions—including 2PC, 3PC, TCC, Saga, local message tables, and MQ‑based approaches—detailing their mechanisms, trade‑offs, and practical implementations with concrete examples from order and cart services.
This article details the challenges of scaling RabbitMQ, the evaluation of RocketMQ versus Pulsar, the architectural design of a new high‑availability message middleware platform, and the step‑by‑step smooth migration strategy that enables higher throughput, richer features, and lower operational costs.
vivo’s Internet Middleware Team replaced its RabbitMQ service with a RocketMQ‑based platform, building an AMQP‑Proxy gateway and metadata layer to enable seamless, zero‑downtime migration while achieving over 100,000 TPS, billion‑message capacity, 50% resource savings, and advanced features such as transactions, ordered and delayed messaging, and tracing.
This article explains the concept, scenarios, advantages, and implementation details of Apache RocketMQ's timed/delay messages, including the TimerWheel algorithm, service workflow, a financial payment‑timeout use case, and practical steps for creating and consuming timed messages.
This article surveys common delayed‑message solutions—including database, RocksDB, Redis, and open‑source MQs like RocketMQ, Pulsar, and QMQ—explaining their architectures, advantages, drawbacks, and practical considerations for building reliable distributed asynchronous messaging.
This article explains the fundamentals of message queues, compares Kafka, RabbitMQ, and RocketMQ in terms of architecture, key concepts, advantages, and drawbacks, and provides practical guidance on selecting the most suitable queue for different application scenarios such as high‑throughput logging, financial transactions, or small‑scale services.
The article examines five backend techniques for automatically closing unpaid e‑commerce orders—scheduled tasks, RocketMQ delayed messages, RabbitMQ dead‑letter queues, a time‑wheel algorithm, and Redis key‑expiration listeners—detailing their principles, advantages, drawbacks, and sample Java code implementations.
This article explains the fundamentals of message middleware, covering its architecture, components, communication models, advantages, common protocols, major open‑source solutions, detailed feature comparisons, and practical guidance for selecting the most suitable message queue based on system scale, business needs, performance, high availability, and operational considerations.
This article explains how RocketMQ’s ordinary messages meet the core demands of business integration, detailing their architecture, lifecycle, key features, typical use cases, and practical code examples for sending and consuming messages.
This article provides a comprehensive introduction to three major message‑queue systems—RabbitMQ, RocketMQ, and Kafka—explaining their core components, exchange types, delivery guarantees, confirmation mechanisms, dead‑letter handling, high‑availability designs, load‑balancing strategies, and common troubleshooting techniques.
