The article explains why adopting a message‑queue improves microservice decoupling and responsibility boundaries, then details the design of a pluggable Base‑MQ component—including @MQEventListener, MQEvent base class, configuration, and MQClient interface—followed by concrete implementations for RocketMQ, Kafka, Redis and RabbitMQ, an example event flow, and guidance for extending the framework.
The article thoroughly explains RocketMQ’s architecture—including stateless NameServers for routing, master‑slave Brokers with sequential CommitLog storage, stateless Producers and pull‑or‑push Consumers, message flow, indexing, and synchronous double‑write reliability—showing how these components deliver high‑throughput, scalable, fault‑tolerant messaging for distributed systems.
This article explains RocketMQ’s batch processing concepts, walks through the early batch model, the index‑build pipeline refactor, the BatchCQ and AutoBatch models, and presents benchmark results that show dramatic throughput gains and lower latency compared to Kafka.
RocketMQ stores messages in a sequential CommitLog while using a ConsumerQueue index for fast random access, and employs two looping threads—RebalanceService for load‑balancing queue assignments and PullMessageService for continuously pulling messages and dispatching them to user callbacks via a thread‑pooled consumer pipeline.
The article explains how RocketMQ’s reliance on mmap‑based zero‑copy leads to more data copies and system‑call overhead compared to Kafka’s sendfile approach, resulting in lower throughput, and discusses when to choose each message‑queue system based on functional needs and performance requirements.
This article explains how RocketMQ achieves high throughput by using sequential disk writes, OS page caching, asynchronous flushing, and zero‑copy techniques, detailing each mechanism and providing code examples. It also discusses the impact on disk seek time and data durability.
This article analyzes the root causes of message loss and duplication in modern message queues, explains how synchronous and asynchronous sending affect reliability, compares storage strategies like sync‑flush and broker clustering, and offers concrete idempotent handling techniques using database constraints and Redis.
This article details the practical journey of building and scaling an Elasticsearch‑based search system that supports tens of millions to billions of items, covering architecture design, capacity planning, multi‑data‑center deployment, data synchronization via RocketMQ and Flink, and multi‑layer reconciliation to ensure consistency and high QPS.
The article outlines why reading source code matters—interview prep, skill growth, design insight—and presents a step‑by‑step guide of 18 practical habits, from mastering the JDK and design patterns to exploring demos, focusing on purpose, main flows, and thoughtful annotation.
This article explores seven classic message‑queue use cases—from asynchronous decoupling and traffic‑shaping to delayed tasks, broadcast consumption, distributed transactions, and using Kafka as a data hub—illustrated with real‑world examples and code snippets.
This article shares seven practical scenarios—such as asynchronous processing, traffic smoothing, message bus, delayed tasks, broadcast consumption, distributed transactions, and data hub integration—where message queues like RocketMQ, Kafka, ActiveMQ, and RabbitMQ help solve high‑concurrency challenges in modern backend systems.
This article investigates the mysterious "prevent gc" comment in RocketMQ source code, explains how Thread.sleep(0) interacts with JVM safepoints and counted loops, and demonstrates practical fixes by changing loop indices or inserting explicit safepoint calls.
This article explains how Java's Thread.sleep(0) works, its native implementation, why it is used in RocketMQ source code, and how the call influences JVM garbage‑collection, safe‑point insertion, and CPU scheduling to improve multithreaded performance.
This article provides a detailed overview of RocketMQ, covering its core concepts, four primary usage scenarios, architectural components such as NameServer, Broker, Producer, and Consumer, as well as message storage, ordering guarantees, high availability, fault tolerance, and transactional messaging.
This article walks through every stage of a RocketMQ message—from producer creation, routing, queue selection, and storage with zero‑copy techniques, through high‑availability replication, consumption modes, ordering guarantees, and finally automatic cleanup—providing code examples and architectural diagrams for each step.
This article compares six practical backend approaches—JDK DelayQueue, RocketMQ delayed messages, Redis expiration, Redis Zset, a dedicated task center, and passive closure—to automatically close unpaid e‑commerce orders after a configurable timeout, outlining their advantages, drawbacks, and suitable scenarios.
To simplify RocketMQ usage, I designed an internal SDK that abstracts complex client configurations, offering single‑call ordered message sending, per‑message consumption, unified result handling, and enforced consistent subscription settings, thereby reducing developer effort and preventing common integration errors.
This article examines why messages can be lost or duplicated in typical queue systems, explains the failure points from producer to broker to consumer, and provides practical techniques such as synchronous flushing, broker clustering, database unique constraints, and Redis deduplication to achieve reliable, idempotent processing.
This article provides a comprehensive overview of four major message queue systems—Kafka, RabbitMQ, RocketMQ, and ActiveMQ—covering their fundamentals, architectures, key concepts, advantages, disadvantages, and guidance for selecting the most suitable solution based on throughput, reliability, scalability, and use‑case requirements.
This article provides a comprehensive overview of Apache RocketMQ, covering its core concepts, message models, reliability mechanisms, ordered message challenges and solutions, as well as detailed explanations of transactional messages, including implementation principles, producer and broker workflows, and practical considerations for ensuring consistency in distributed systems.
This article compares four distributed transaction solutions—local message tables, RocketMQ transactional messages, Seata AT mode, and Seata TCC mode—detailing their advantages, limitations, workflow phases, and common exception‑handling techniques for ensuring data consistency across services.
This article explains the motivations behind RocketMQ ACL 2.0, outlines its six major enhancements—including fine‑grained API permissions, flexible matching modes, and cluster‑wide access control—details the RBAC/ABAC model, authentication and authorization workflows, configuration examples, command‑line usage, and migration strategies, and discusses future planning for the access control system.
This article recounts how a volunteer registration platform was redesigned to meet extreme concurrency demands—handling up to 40k QPS and 20k TPS—by evaluating MySQL limits, adding Redis caching, integrating RocketMQ, applying fault‑recovery mechanisms, and iteratively optimizing through multiple load‑testing rounds.
This article provides a comprehensive overview of Apache RocketMQ, detailing its core components, producer and consumer workflows, storage strategies, master‑slave synchronization, Raft‑based half‑write and leader election mechanisms, and best‑practice recommendations for high‑throughput, fault‑tolerant messaging systems.
This article provides a detailed introduction to RocketMQ, covering its purpose as a distributed messaging middleware, the roles of NameServer, Producer, Broker, and Consumer, and explains how each component functions within a scalable, high‑throughput architecture.
This article explores the evolution of message middleware, defines event‑driven architecture, explains why it resurfaces in the cloud era, and details RocketMQ 5.0 EventBridge’s design—its CloudEvents‑based abstraction, schema support, rule engine, observability, and real‑world use cases—offering practical guidance for building scalable, decoupled systems.
This article explains the rapid growth of IoT, the unique messaging requirements it creates, compares classic and IoT message scenarios, and details how RocketMQ 5.0’s MQTT sub‑product introduces a lightweight protocol, read‑amplify storage, edge‑cloud integration, massive queue support, and a push‑pull delivery model to meet cloud‑native IoT demands.
This article shares seven classic use cases of message queues—including asynchronous decoupling, traffic smoothing, bus architecture, delayed tasks, broadcast consumption, distributed transactions, and data hub integration—illustrated with real-world experiences and code examples to help engineers design robust high‑concurrency systems.
From asynchronous decoupling to distributed transactions, this article explores seven classic message‑queue use cases—such as peak‑shaving, event buses, delayed tasks, broadcast consumption, and data hub integration—illustrated with real‑world examples from e‑commerce, ride‑hailing, and lottery systems, plus RocketMQ code snippets.
The article details how vivo’s Internet Middleware Team migrated from RabbitMQ to RocketMQ and introduced a user‑property‑based multi‑version isolation layer that tags messages, filters them by consumer groups, and provides fallback and retry mechanisms, enabling test‑environment separation without SDK changes, though offline consumers may miss baseline messages.
This article recounts the end‑to‑end design, bottleneck analysis, and iterative performance tuning—including MySQL, Redis, RocketMQ, compression, sharding, and connection‑pool adjustments—that enabled a volunteer registration platform to meet demanding high‑concurrency and data‑accuracy requirements under limited resources.
This article explains the evolution of message middleware, introduces core concepts of stream processing, and details RocketMQ 5.0's native lightweight stream engine RStreams and its stream database RSQLDB, showing how they simplify real‑time data integration, computation, and scaling in cloud‑native environments.
This article recounts the end‑to‑end design, testing, and iterative optimization of a volunteer registration platform that required tens of thousands of QPS, covering requirement analysis, middleware benchmarking, data partitioning, compression, connection‑limit handling, and final deployment lessons.
This article walks through the end‑to‑end design and implementation of a searchable system that scales from millions to hundreds of millions of products, covering business background, data‑center architecture, ES cluster sizing, multi‑room data sync, RocketMQ‑Flink pipelines, and multi‑layer reconciliation to achieve high QPS and strong consistency.
This article analyzes multiple engineering solutions for automatically cancelling orders after the payment timeout, covering single‑machine and cluster scheduled‑task designs (Quartz, ElasticJob, XXL‑JOB), delayed‑message techniques (RocketMQ, custom delay service, Redis), and essential concurrency and monitoring best practices.
This article explores the ten key architectural and implementation techniques—such as batch messaging, compression, Netty networking, zero‑copy I/O, sequential writes, efficient storage structures, asynchronous processing, batch handling, lock optimizations, and thread‑pool isolation—that together make RocketMQ a high‑performance message middleware.
This article breaks down the ten key mechanisms—batch sending, message compression, Netty‑based networking, zero‑copy I/O, sequential writes, optimized storage structures, asynchronous flushing and replication, batch processing, lock refinements, and thread‑pool isolation—that together give RocketMQ its remarkable speed and efficiency.
This guide walks you through setting up the Java environment, downloading Apache RocketMQ 5.1.4, installing its NameServer, Broker, and optional Proxy components on an Ubuntu 20.04 cloud instance, and configuring both local and cluster deployment modes with optional visual dashboard tools.
This guide explains why RocketMQ consumer retries can be slow under HTTP, outlines the retry mechanism and limits, and provides step‑by‑step instructions with screenshots for manually resending messages via the console to troubleshoot system anomalies.
Dead Letter QueueHTTP ProtocolManual Message Resend
This article explains Java's CompletableFuture, its advantages over the traditional Future API, demonstrates common methods with code examples, and shows how RocketMQ leverages CompletableFuture to coordinate asynchronous disk flush and replica synchronization tasks.
This article provides an in‑depth introduction to RocketMQ, covering its evolution, core components, message domain model, key characteristics such as ordered delivery and deduplication, and typical use cases like traffic shaping, asynchronous decoupling, and distributed transaction messaging.
This article explains the fundamentals, architectures, key concepts, and trade‑offs of four popular message‑queue systems—Kafka, RabbitMQ, RocketMQ and ActiveMQ—providing practical guidance for interview preparation and technology selection in modern backend development.
This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.
This article demonstrates how to efficiently build department and regional hierarchical trees in Java by fetching all records in a single database query, using Stream API, Lambda expressions, in‑memory grouping, Redis caching and RocketMQ for asynchronous updates, achieving sub‑second response times even with tens of thousands of nodes.
This article introduces five widely used message middleware systems—Kafka, RabbitMQ, ActiveMQ, RocketMQ, and Pulsar—explaining their core architectures, key components, typical use cases, and advantages for building reliable, high‑throughput backend solutions.
This article explains what message middleware is, outlines its key use cases such as asynchronous communication and decoupling, details core principles and models like point‑to‑point and publish/subscribe, reviews popular solutions (Kafka, RabbitMQ, RocketMQ, Pulsar, etc.), and offers selection guidance.
This article explains why MQ is needed beyond in‑process event listeners, details the required dependencies for Spring Boot 3, shows how to configure RocketMQ, provides code snippets for producer, consumer, and testing, and offers practical tips for handling async, batch, and one‑way messaging.
This article records a complete, step‑by‑step installation and startup process for Apache RocketMQ on a CentOS 7 system, covering environment checks, downloading the package, extracting files, launching the name server and broker, adjusting memory settings, and verifying the setup with a test client.
This article provides a comprehensive overview of Spring Cloud Alibaba’s core features—including Nacos service registration and configuration, Sentinel flow‑control and degradation rules, Seata distributed transaction modes, RocketMQ messaging capabilities, as well as guidance on service discovery, security, gray releases, and dynamic routing.
This article explains why RocketMQ messages can pile up, analyzes the consumer pull‑process bottlenecks of consumption latency and concurrency, and provides practical strategies—including logging, JVM stack inspection, and concurrency tuning—to identify and eliminate the root causes of message backlog.
The paper presents a centralized online batch‑processing framework for large‑scale promotion systems, where applications integrate via an SDK, a task‑center schedules and dispatches sub‑tasks through RocketMQ to Dubbo‑enabled containers, employing MapReduce‑style splitting, Guava rate‑limiting, heartbeat health checks, and has successfully handled over 1.3 million tasks during Double‑11.
This article explains RocketMQ's random indexing mechanism, detailing its on‑disk three‑segment hash table structure, the compact format conversion process, multi‑threaded write and query workflows, layered system design, crash‑recovery strategy, and comparisons with RocksDB and InnoDB storage engines.
This article explains the principles of distributed transaction messages, comparing 2PC, TCC, and transactional messaging, and provides detailed walkthroughs of RocketMQ and Kafka implementations, including their two‑phase processes, broker handling, and source‑code insights for ensuring data consistency in asynchronous systems.
This article explains why load balancing is crucial for RocketMQ producers, details the internal selection algorithm using ThreadLocal indexes and round‑robin logic, and provides Java code examples illustrating how messages are distributed across queues.
This comprehensive article explains the fundamentals of message‑queue middleware, covering point‑to‑point and publish/subscribe models, key evaluation metrics, detailed comparisons of RabbitMQ, Kafka, RocketMQ and ActiveMQ, and deep dives into Kafka and RocketMQ architectures, features, and best‑practice usage.
This article explains the concept of a local message table, shows how it can be used to achieve reliable distributed transaction consistency with MQ, and provides complete MySQL DDL and Java service code along with a retry mechanism and analysis of failure scenarios.
The author celebrates Programmer's Day by sharing a comprehensive 100,000‑word e‑book on RocketMQ design, detailing personal learning experiences, practical applications, and key concepts to help backend engineers quickly master this powerful message‑queue technology.
This article explains RocketMQ's core components, the differences between push and pull consumption modes, and how the framework implements a pseudo‑push mechanism using long polling, including detailed code analysis of the consumer and broker side processes.
This article explains when adding consumers to a RocketMQ topic improves message consumption, the reasons for consumer pull delays, and details six MessageQueue allocation strategies—including average, round‑robin, custom, machine‑room, nearby, and consistent‑hash—accompanied by code examples.
This article explains the core components of RocketMQ—including NameServer, Broker, Producer, and Consumer—detailing their functions, deployment diagrams, and how the middleware supports high‑concurrency scenarios such as peak shaving, asynchronous messaging, fault tolerance, and system decoupling.
This article explains how Apache RocketMQ performs server‑side message filtering using Tag and SQL92 expressions, outlines the workflow for producers and consumers, compares the two methods, and provides Java code examples for implementing both simple tag filtering and advanced SQL‑based attribute filtering.
This guide walks you through using Alibaba Cloud PTS and JMeter 5.5 to create a custom JavaSampler for RocketMQ 5.0, package the sampler into a JAR, configure a PTS JMeter environment, run an RPS‑mode load test, and analyze the detailed performance report.
This article explains why batch messaging in RocketMQ is crucial for high-throughput systems, outlines its characteristics and ideal scenarios, and provides complete Java code examples for sending and splitting large batches to improve performance and reduce downstream API calls.
This article explains how to apply gray release techniques in RocketMQ by upgrading consumers, upgrading producers, and using consumer‑side filtering, tag filtering, and SQL92 filtering, providing concrete code examples, configuration steps, and trade‑off analysis for each approach.
This article explains seven typical message‑queue use cases—including ordinary, ordered, delayed, transactional, trace, dead‑letter, and priority messages—detailing their business motivations, implementation challenges, and concrete code examples for Kafka, RocketMQ, Pulsar, and RabbitMQ.
This course provides an in‑depth study of RocketMQ, covering its fundamentals, source‑code analysis, advanced features such as transactional and ordered messages, performance optimization, operational monitoring, and hands‑on projects that guide learners through installation, configuration, clustering, and integration with Spring Boot.
This article examines the numerous scheduled tasks in RocketMQ—covering architecture, producer and consumer maintenance, broker housekeeping, and NameServer management—to reveal how periodic jobs ensure reliable message handling, performance monitoring, and system self‑protection.
This article explains the various scheduled tasks in RocketMQ, covering architecture overview, producer and consumer periodic operations, broker maintenance jobs, and NameServer housekeeping, while providing concrete Java code examples to illustrate how each task contributes to message handling, monitoring, and system reliability.
Learn step‑by‑step how to configure Alibaba’s open‑source Canal to capture MySQL binlog changes, route them through RocketMQ, and index the data into Elasticsearch, covering cluster mode, MySQL and Elasticsearch setup, Canal properties, and consumer implementation details.
This tutorial walks through configuring Canal in cluster mode to capture MySQL row‑based binlog events, publish them to a RocketMQ topic, and consume the ordered messages to update an Elasticsearch index, covering MySQL, RocketMQ, Elasticsearch mappings, and the required Canal properties.
This article explains RocketMQ's internal workflow for concurrent and ordered message consumption, covering thread‑pool configuration, pull‑message handling, request submission, execution steps, offset management, expired‑message cleanup, and answers common questions about batch retries and offset ordering.
Explore how Java's CompletableFuture enhances asynchronous programming by overcoming Future's limitations, learn its core APIs, practical code examples, and see a real-world application in RocketMQ where combined futures streamline message persistence and replication.
From the early days of ActiveMQ and RabbitMQ to the high‑throughput designs of Kafka, RocketMQ, and Pulsar, this article traces the development stages of message queues, explains core concepts such as topics, producers, consumers, and partitions, and compares their architectural and storage designs.
This tutorial demonstrates how to use Alibaba's Canal in cluster mode with Zookeeper to capture MySQL row‑based binlog events, publish them via RocketMQ, and consume them with a custom listener that transforms inserts, updates, and deletes into synchronized documents in an Elasticsearch index.
This step‑by‑step tutorial shows how to configure Alibaba's Canal to capture MySQL binlog changes, route them through RocketMQ, and index the data into Elasticsearch, covering cluster mode, MySQL and Elasticsearch setup, RocketMQ topic creation, Canal properties, and consumer implementation.
This article provides a detailed overview of popular message queue systems, explains their core concepts such as decoupling and eventual consistency, and compares the advantages and disadvantages of Kafka, RocketMQ, RabbitMQ, and other notable MQ solutions for high‑concurrency scenarios.
This guide demonstrates how to integrate RocketMQ 4.8.0 into a Spring Boot 2.4.12 application, covering dependency setup, configuration, sending and receiving normal, ordered, cluster/broadcast, and transactional messages, complete with code snippets and execution results to help developers implement reliable messaging patterns.
This article explains the three core benefits of message queues, compares major MQ products, details RocketMQ's advantages, internal models, reliability mechanisms, load‑balancing, storage design, and practical code examples, guiding engineers to make informed architectural decisions.
This article explains RocketMQ's advanced features—transactional messaging, ordered messaging, SQL‑based filtering, delayed messaging, and global high‑availability—detailing their principles, implementation steps, practical demos, and how they solve consistency challenges in large‑scale e‑commerce systems.
This article explains RocketMQ's master‑slave replication mechanism, comparing synchronous and asynchronous modes, detailing metadata and message data copying processes, and showing how synchronous guarantees are implemented using CompletableFuture to ensure high availability.
This guide explains how to achieve reliable message consistency in a Spring Boot 2.3.9 application using RocketMQ 4.8.0 by outlining the transactional message flow, broker handling, project structure, code implementation, and testing procedures.
This article explains RocketMQ transaction messages by covering their use cases, underlying mechanisms, and a step‑by‑step implementation that demonstrates how to ensure eventual consistency between message production and local database transactions in distributed Java back‑end systems.
This article provides a comprehensive walkthrough of RocketMQ 4.9.x consumption architecture, covering the four core roles, publish‑subscribe model, storage structures, load‑balancing, long‑polling, concurrent and ordered consumption flows, progress persistence, and retry mechanisms, with illustrative diagrams and code snippets.
This article explains the ACID properties of transactions and compares various implementation approaches—including database‑level mechanisms, Spring single‑datasource transactions, and distributed transaction frameworks such as Seata and RocketMQ—while analyzing their advantages, drawbacks, and suitable scenarios.
After a massive message backlog in RocketMQ, the author discovered that identical clientIds generated by Docker containers using host network mode caused broker load‑balancing errors, leading to slow consumption; the article explains the root cause, code analysis, and how setting a unique clientId resolves the issue.
This article traces the development of message queues from early decoupling solutions like ActiveMQ and RabbitMQ, through high‑throughput designs such as Kafka and RocketMQ, to modern platform‑centric systems like Pulsar, while detailing core concepts, architecture diagrams, storage mechanisms and trade‑offs.
This article examines the false empty response issue caused by long‑polling coupling in RocketMQ 5.0 SimpleConsumer, analyzes existing approaches, and presents an adaptive, stateless proxy solution that decouples client and broker polls, adds notification, and includes metrics and configuration guidance.
The article traces two decades of message‑queue evolution—from early decoupling tools like ActiveMQ, through Kafka’s high‑throughput log model, to Pulsar’s layered, cloud‑native architecture—explaining core concepts, storage designs, and trade‑offs that guide choosing the right MQ for modern workloads.
This article explores RocketMQ's high availability mechanism, focusing on DLedger's implementation of Raft-based leader election for distributed message middleware, covering both traditional and Controller mode architectures.
This guide explains why and how to read open‑source Java code, covering essential JDK knowledge, design patterns, official documentation, module structure, demo‑first approaches, purposeful reading strategies, class‑name clues, class hierarchy analysis, commenting practices, and leveraging related resources.
This article provides a comprehensive overview of RocketMQ, covering its origin, core concepts, component roles, cluster deployment architecture, workflow, feature details, persistence mechanisms, and cleanup policies, offering developers a solid foundation for using this high‑performance messaging middleware.
This article examines RocketMQ's high‑availability evolution—from early master‑slave and Raft‑based designs to the v5 DLedger fusion model—detailing replica groups, data sharding, election mechanisms, replication strategies, metric trade‑offs, log‑divergence handling, controller roles, heartbeat optimizations, and comparisons with Kafka and Pulsar, all illustrated with diagrams and code snippets.
This article walks through RocketMQ 4.9.x’s consumption architecture, explaining the roles of NameServer, Broker, Producer and Consumer, the publish‑subscribe model, storage structures, load‑balancing algorithms, long‑polling, concurrent and ordered consumption, progress persistence, and the built‑in retry mechanism.
This article explains the architecture of a payment center that unifies internal payment services and external third‑party integrations, details the three‑step payment flow, discusses common issues such as order timeout and result reliability, and demonstrates how template‑method and strategy patterns together with RocketMQ delay queues solve these problems in a SpringBoot backend.
The article explains when adding consumers can reduce RocketMQ backlog, the reasons for delayed message pulling, handling slow external services, and details six different MessageQueue allocation strategies—including average, round‑robin, custom, machine‑room, nearby, and consistent‑hash—accompanied by Java code examples.
This article explains why ordered message processing is crucial for business workflows, outlines how to maintain order during production, storage, and consumption in RocketMQ, and describes the locking mechanisms required to achieve reliable sequential consumption.
RocketMQ 5.1.0 introduces a tiered storage module that offloads messages from local disks to cheaper storage, enabling longer retention without impacting hot data performance, and the article explains its design, implementation details, configuration steps, and current challenges.
The article explains how RocketMQ 5.1.0 introduces a tiered storage module that offloads messages to cheaper media, describes its design, architecture layers, quick‑start configuration, upload and read mechanisms, prefetch cache, fault recovery, current development plans, and remaining challenges.