This article explores eight practical use cases for message queues—including asynchronous processing, service decoupling, traffic shaping, delayed tasks, log collection, distributed transactions, remote calls, and broadcast notifications—providing code examples and architectural guidance for building robust backend systems.
This article explains eight typical scenarios for using message queues—including asynchronous processing, service decoupling, traffic shaping, delayed tasks, log collection, distributed transactions, remote invocation, and broadcast notifications—providing clear explanations and Java code snippets for each case.
The article investigates a mysterious "prevent gc" comment in RocketMQ's source, explains how Thread.sleep(0) forces a JVM safepoint to mitigate long GC pauses, and demonstrates that changing loop counters from int to long eliminates the need for the hack.
This article explains the architectural changes introduced in RocketMQ 5.0, including the new stateless Proxy layer, the POP consumption model, message‑level load balancing, and the Controller mode for automatic master‑slave failover, providing detailed diagrams and resource links.
This article explains RocketMQ’s fundamental architecture, detailing the definitions, roles, and interactions of topics, message queues, messages, producers, consumers, consumer groups, and subscription relationships, and highlights how these components work together to achieve high performance, availability, and scalability.
This article explains the limitations of Java's Future, introduces CompletableFuture with its rich API for non‑blocking asynchronous programming, demonstrates practical usage including task creation, result retrieval, chaining, exception handling, and shows how RocketMQ leverages CompletableFuture to coordinate disk flush and replica sync tasks efficiently.
This article examines the causes of message queue disorder in distributed systems and presents practical solutions such as ordered messaging, pre‑consumption checks, state machines, and monitoring to improve system stability and data consistency.
This article explains Apache RocketMQ's core components—including producers, topics, queues, and consumer groups—covers synchronous RPC versus asynchronous messaging, compares point‑to‑point and publish‑subscribe transmission models, and highlights their suitable scenarios and trade‑offs.
The article explains how Kafka’s use of sendfile zero‑copy gives it higher throughput than RocketMQ, which relies on mmap, and discusses the trade‑offs between performance and feature richness when choosing between the two message‑queue systems.
This comprehensive guide introduces RocketMQ's architecture, compares it with RabbitMQ and Kafka, outlines typical use cases, explains key concepts such as producers, brokers, consumers, topics, tags, and offsets, and provides complete Java code examples for building producers and consumers.
This article explains the motivation, design choices, architecture, and core technical details of implementing the AMQP protocol on top of RocketMQ, including model mapping, proxy layer, LMQ queues, message flow, binary protocol handling, feature comparison, performance tuning, and future plans.
The article compares Kafka and RocketMQ, explaining how Kafka’s use of sendfile zero‑copy reduces system calls and data copies, while RocketMQ relies on mmap, leading to lower throughput but richer features, and offers guidance on choosing between them based on scenario.
An unexpected RocketMQ alert revealed over 300 million queued messages, traced to identical clientIds generated by Docker host‑network containers; the article explains the root cause, examines clientId generation, load‑balancing logic, and provides a fix by customizing the clientId to prevent message backlog.
This article explains RocketMQ's transactional message mechanism, covering half‑message concepts, back‑check processes, implementation steps with code examples, and how to achieve reliable event handling and distributed transaction consistency in a microservice architecture.
A deep dive into a massive RocketMQ message backlog reveals that identical clientIds generated in Docker host‑mode cause load‑balancing errors, and the article explains the root cause, code analysis, and a practical fix by customizing the clientId.
Xiaohongshu replaced its RocketMQ‑based online messaging platform with Apache Pulsar, achieving up to 48% total cost reduction, 43% higher CPU utilization, 30% resource savings, and a latency drop from 20.2 ms to 5.7 ms through cloud‑native, elastic scaling and a phased migration strategy.
A recent CCF‑A conference paper reveals that an adaptive K‑value backoff lock, derived from queueing theory and implemented in Apache RocketMQ, can replace both spin and mutex locks, achieving up to 37.58% performance gains on x86 CPUs and 32.82% on ARM while reducing CPU usage and resource consumption.
This article provides a comprehensive overview of RocketMQ, covering its architecture, key features such as high reliability, low latency and high throughput, comparisons with Kafka, RabbitMQ and ActiveMQ, and detailed mechanisms that ensure message durability, performance, and ordered consumption.
This article explains how RocketMQ guarantees message reliability across the production, storage, and consumption stages, covering synchronous, asynchronous, and transactional sending, disk persistence and replication strategies, consumption models (at-least-once, at-most-once, exactly-once), and handling consumer crashes without message loss.
The article investigates why a RocketMQ consumer service running on a 4‑core ECS experiences sustained high load despite low CPU, I/O and memory usage, identifies excessive thread creation and frequent trace‑module context switches as the main causes, and proposes configuration and SDK upgrades to resolve the issue.
This comprehensive guide compares five popular message queue systems—Kafka, Pulsar, RabbitMQ, RocketMQ, and NSQ—covering their architectures, core concepts, selection criteria, feature differences such as ordering, latency, durability, scaling, and operational considerations, to help engineers select the most suitable solution for their use cases.
When a SMS service fails and messages need to be re‑sent, this article shows how to use RocketMQ's message backtracking feature to reset consumer offsets, verify the reset, and explore the internal workflow and alternative pull‑consumer strategies with concrete code examples and detailed analysis.
The article surveys major message‑queue systems—Kafka, Pulsar, RocketMQ, RabbitMQ, and NSQ—detailing their architectures, key features, selection criteria such as ordering, scalability and latency, performance benchmarks, operational considerations, and typical use cases, and advises which platform best fits high‑throughput, reliability, low‑latency, or lightweight Go‑centric scenarios.
The article chronicles the author’s evolution with message queues—from early experiments with ActiveMQ, through Redis and RabbitMQ, to MetaQ and RocketMQ—highlighting real‑world challenges, architectural decisions, scaling tactics, and practical insights for building robust asynchronous systems.
This article explains how RocketMQ's transaction messaging can break tight coupling between heterogeneous systems in fund companies, outlines topic/tag classification, compares distributed‑transaction solutions, and details the end‑to‑end workflow and consumption guarantees for financial use cases.
This article shares seven classic message‑queue use cases—from asynchronous decoupling and peak‑smoothing to delayed tasks, broadcast consumption, distributed transactions, and data hub integration—illustrated with real‑world examples, diagrams, and RocketMQ code snippets.
This article explains RocketMQ's core components, including domain models, message transmission models, reliability mechanisms, the challenges of ordered messaging, practical usage scenarios, and the design and implementation of transactional messages in both open‑source and proprietary versions.
This article explains how to use RocketMQ's message backtracking feature to reset consumer offsets, covering problem background, verification steps, core process analysis, code examples, and comparative evaluation of different approaches in real-world scenarios.
The article explains why adopting a message‑queue improves microservice decoupling and responsibility boundaries, then details the design of a pluggable Base‑MQ component—including @MQEventListener, MQEvent base class, configuration, and MQClient interface—followed by concrete implementations for RocketMQ, Kafka, Redis and RabbitMQ, an example event flow, and guidance for extending the framework.
The article thoroughly explains RocketMQ’s architecture—including stateless NameServers for routing, master‑slave Brokers with sequential CommitLog storage, stateless Producers and pull‑or‑push Consumers, message flow, indexing, and synchronous double‑write reliability—showing how these components deliver high‑throughput, scalable, fault‑tolerant messaging for distributed systems.
This article explains RocketMQ’s batch processing concepts, walks through the early batch model, the index‑build pipeline refactor, the BatchCQ and AutoBatch models, and presents benchmark results that show dramatic throughput gains and lower latency compared to Kafka.
RocketMQ stores messages in a sequential CommitLog while using a ConsumerQueue index for fast random access, and employs two looping threads—RebalanceService for load‑balancing queue assignments and PullMessageService for continuously pulling messages and dispatching them to user callbacks via a thread‑pooled consumer pipeline.
The article explains how RocketMQ’s reliance on mmap‑based zero‑copy leads to more data copies and system‑call overhead compared to Kafka’s sendfile approach, resulting in lower throughput, and discusses when to choose each message‑queue system based on functional needs and performance requirements.
This article explains how RocketMQ achieves high throughput by using sequential disk writes, OS page caching, asynchronous flushing, and zero‑copy techniques, detailing each mechanism and providing code examples. It also discusses the impact on disk seek time and data durability.
This article analyzes the root causes of message loss and duplication in modern message queues, explains how synchronous and asynchronous sending affect reliability, compares storage strategies like sync‑flush and broker clustering, and offers concrete idempotent handling techniques using database constraints and Redis.
This article details the practical journey of building and scaling an Elasticsearch‑based search system that supports tens of millions to billions of items, covering architecture design, capacity planning, multi‑data‑center deployment, data synchronization via RocketMQ and Flink, and multi‑layer reconciliation to ensure consistency and high QPS.
This article explores seven classic message‑queue use cases—from asynchronous decoupling and traffic‑shaping to delayed tasks, broadcast consumption, distributed transactions, and using Kafka as a data hub—illustrated with real‑world examples and code snippets.
This article shares seven practical scenarios—such as asynchronous processing, traffic smoothing, message bus, delayed tasks, broadcast consumption, distributed transactions, and data hub integration—where message queues like RocketMQ, Kafka, ActiveMQ, and RabbitMQ help solve high‑concurrency challenges in modern backend systems.
This article investigates the mysterious "prevent gc" comment in RocketMQ source code, explains how Thread.sleep(0) interacts with JVM safepoints and counted loops, and demonstrates practical fixes by changing loop indices or inserting explicit safepoint calls.
This article explains how Java's Thread.sleep(0) works, its native implementation, why it is used in RocketMQ source code, and how the call influences JVM garbage‑collection, safe‑point insertion, and CPU scheduling to improve multithreaded performance.
This article provides a detailed overview of RocketMQ, covering its core concepts, four primary usage scenarios, architectural components such as NameServer, Broker, Producer, and Consumer, as well as message storage, ordering guarantees, high availability, fault tolerance, and transactional messaging.
This article walks through every stage of a RocketMQ message—from producer creation, routing, queue selection, and storage with zero‑copy techniques, through high‑availability replication, consumption modes, ordering guarantees, and finally automatic cleanup—providing code examples and architectural diagrams for each step.
This article compares six practical backend approaches—JDK DelayQueue, RocketMQ delayed messages, Redis expiration, Redis Zset, a dedicated task center, and passive closure—to automatically close unpaid e‑commerce orders after a configurable timeout, outlining their advantages, drawbacks, and suitable scenarios.
To simplify RocketMQ usage, I designed an internal SDK that abstracts complex client configurations, offering single‑call ordered message sending, per‑message consumption, unified result handling, and enforced consistent subscription settings, thereby reducing developer effort and preventing common integration errors.
This article examines why messages can be lost or duplicated in typical queue systems, explains the failure points from producer to broker to consumer, and provides practical techniques such as synchronous flushing, broker clustering, database unique constraints, and Redis deduplication to achieve reliable, idempotent processing.
This article provides a comprehensive overview of four major message queue systems—Kafka, RabbitMQ, RocketMQ, and ActiveMQ—covering their fundamentals, architectures, key concepts, advantages, disadvantages, and guidance for selecting the most suitable solution based on throughput, reliability, scalability, and use‑case requirements.
This article provides a comprehensive overview of Apache RocketMQ, covering its core concepts, message models, reliability mechanisms, ordered message challenges and solutions, as well as detailed explanations of transactional messages, including implementation principles, producer and broker workflows, and practical considerations for ensuring consistency in distributed systems.
This article compares four distributed transaction solutions—local message tables, RocketMQ transactional messages, Seata AT mode, and Seata TCC mode—detailing their advantages, limitations, workflow phases, and common exception‑handling techniques for ensuring data consistency across services.
This article explains the motivations behind RocketMQ ACL 2.0, outlines its six major enhancements—including fine‑grained API permissions, flexible matching modes, and cluster‑wide access control—details the RBAC/ABAC model, authentication and authorization workflows, configuration examples, command‑line usage, and migration strategies, and discusses future planning for the access control system.
This article recounts how a volunteer registration platform was redesigned to meet extreme concurrency demands—handling up to 40k QPS and 20k TPS—by evaluating MySQL limits, adding Redis caching, integrating RocketMQ, applying fault‑recovery mechanisms, and iteratively optimizing through multiple load‑testing rounds.
This article provides a comprehensive overview of Apache RocketMQ, detailing its core components, producer and consumer workflows, storage strategies, master‑slave synchronization, Raft‑based half‑write and leader election mechanisms, and best‑practice recommendations for high‑throughput, fault‑tolerant messaging systems.
This article provides a detailed introduction to RocketMQ, covering its purpose as a distributed messaging middleware, the roles of NameServer, Producer, Broker, and Consumer, and explains how each component functions within a scalable, high‑throughput architecture.
This article explores the evolution of message middleware, defines event‑driven architecture, explains why it resurfaces in the cloud era, and details RocketMQ 5.0 EventBridge’s design—its CloudEvents‑based abstraction, schema support, rule engine, observability, and real‑world use cases—offering practical guidance for building scalable, decoupled systems.
This article explains the rapid growth of IoT, the unique messaging requirements it creates, compares classic and IoT message scenarios, and details how RocketMQ 5.0’s MQTT sub‑product introduces a lightweight protocol, read‑amplify storage, edge‑cloud integration, massive queue support, and a push‑pull delivery model to meet cloud‑native IoT demands.
This article shares seven classic use cases of message queues—including asynchronous decoupling, traffic smoothing, bus architecture, delayed tasks, broadcast consumption, distributed transactions, and data hub integration—illustrated with real-world experiences and code examples to help engineers design robust high‑concurrency systems.
From asynchronous decoupling to distributed transactions, this article explores seven classic message‑queue use cases—such as peak‑shaving, event buses, delayed tasks, broadcast consumption, and data hub integration—illustrated with real‑world examples from e‑commerce, ride‑hailing, and lottery systems, plus RocketMQ code snippets.
The article details how vivo’s Internet Middleware Team migrated from RabbitMQ to RocketMQ and introduced a user‑property‑based multi‑version isolation layer that tags messages, filters them by consumer groups, and provides fallback and retry mechanisms, enabling test‑environment separation without SDK changes, though offline consumers may miss baseline messages.
This article recounts the end‑to‑end design, bottleneck analysis, and iterative performance tuning—including MySQL, Redis, RocketMQ, compression, sharding, and connection‑pool adjustments—that enabled a volunteer registration platform to meet demanding high‑concurrency and data‑accuracy requirements under limited resources.
This article explains the evolution of message middleware, introduces core concepts of stream processing, and details RocketMQ 5.0's native lightweight stream engine RStreams and its stream database RSQLDB, showing how they simplify real‑time data integration, computation, and scaling in cloud‑native environments.
This article recounts the end‑to‑end design, testing, and iterative optimization of a volunteer registration platform that required tens of thousands of QPS, covering requirement analysis, middleware benchmarking, data partitioning, compression, connection‑limit handling, and final deployment lessons.
This article walks through the end‑to‑end design and implementation of a searchable system that scales from millions to hundreds of millions of products, covering business background, data‑center architecture, ES cluster sizing, multi‑room data sync, RocketMQ‑Flink pipelines, and multi‑layer reconciliation to achieve high QPS and strong consistency.
This article explores Apache RocketMQ 5.x’s new cloud‑native capabilities—including the Proxy component, gRPC client, Pop consumption model, timer‑based delayed messages, tiered storage, distributed rate‑limiting, and containerized deployment on Tencent Cloud—while outlining architecture changes, practical usage patterns, and future directions.
This article explores the ten key architectural and implementation techniques—such as batch messaging, compression, Netty networking, zero‑copy I/O, sequential writes, efficient storage structures, asynchronous processing, batch handling, lock optimizations, and thread‑pool isolation—that together make RocketMQ a high‑performance message middleware.
This article breaks down the ten key mechanisms—batch sending, message compression, Netty‑based networking, zero‑copy I/O, sequential writes, optimized storage structures, asynchronous flushing and replication, batch processing, lock refinements, and thread‑pool isolation—that together give RocketMQ its remarkable speed and efficiency.
This guide walks you through setting up the Java environment, downloading Apache RocketMQ 5.1.4, installing its NameServer, Broker, and optional Proxy components on an Ubuntu 20.04 cloud instance, and configuring both local and cluster deployment modes with optional visual dashboard tools.
This guide explains why RocketMQ consumer retries can be slow under HTTP, outlines the retry mechanism and limits, and provides step‑by‑step instructions with screenshots for manually resending messages via the console to troubleshoot system anomalies.
This article explains Java's CompletableFuture, its advantages over the traditional Future API, demonstrates common methods with code examples, and shows how RocketMQ leverages CompletableFuture to coordinate asynchronous disk flush and replica synchronization tasks.
This article provides an in‑depth introduction to RocketMQ, covering its evolution, core components, message domain model, key characteristics such as ordered delivery and deduplication, and typical use cases like traffic shaping, asynchronous decoupling, and distributed transaction messaging.
This article explains the fundamentals, architectures, key concepts, and trade‑offs of four popular message‑queue systems—Kafka, RabbitMQ, RocketMQ and ActiveMQ—providing practical guidance for interview preparation and technology selection in modern backend development.
This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.
This article demonstrates how to efficiently build department and regional hierarchical trees in Java by fetching all records in a single database query, using Stream API, Lambda expressions, in‑memory grouping, Redis caching and RocketMQ for asynchronous updates, achieving sub‑second response times even with tens of thousands of nodes.
This article introduces five widely used message middleware systems—Kafka, RabbitMQ, ActiveMQ, RocketMQ, and Pulsar—explaining their core architectures, key components, typical use cases, and advantages for building reliable, high‑throughput backend solutions.
This article explains what message middleware is, outlines its key use cases such as asynchronous communication and decoupling, details core principles and models like point‑to‑point and publish/subscribe, reviews popular solutions (Kafka, RabbitMQ, RocketMQ, Pulsar, etc.), and offers selection guidance.
This article explains why MQ is needed beyond in‑process event listeners, details the required dependencies for Spring Boot 3, shows how to configure RocketMQ, provides code snippets for producer, consumer, and testing, and offers practical tips for handling async, batch, and one‑way messaging.
This article records a complete, step‑by‑step installation and startup process for Apache RocketMQ on a CentOS 7 system, covering environment checks, downloading the package, extracting files, launching the name server and broker, adjusting memory settings, and verifying the setup with a test client.
This article provides a comprehensive overview of Spring Cloud Alibaba’s core features—including Nacos service registration and configuration, Sentinel flow‑control and degradation rules, Seata distributed transaction modes, RocketMQ messaging capabilities, as well as guidance on service discovery, security, gray releases, and dynamic routing.
This article explains why RocketMQ messages can pile up, analyzes the consumer pull‑process bottlenecks of consumption latency and concurrency, and provides practical strategies—including logging, JVM stack inspection, and concurrency tuning—to identify and eliminate the root causes of message backlog.
The paper presents a centralized online batch‑processing framework for large‑scale promotion systems, where applications integrate via an SDK, a task‑center schedules and dispatches sub‑tasks through RocketMQ to Dubbo‑enabled containers, employing MapReduce‑style splitting, Guava rate‑limiting, heartbeat health checks, and has successfully handled over 1.3 million tasks during Double‑11.
This article explains RocketMQ's random indexing mechanism, detailing its on‑disk three‑segment hash table structure, the compact format conversion process, multi‑threaded write and query workflows, layered system design, crash‑recovery strategy, and comparisons with RocksDB and InnoDB storage engines.
This article explains the principles of distributed transaction messages, comparing 2PC, TCC, and transactional messaging, and provides detailed walkthroughs of RocketMQ and Kafka implementations, including their two‑phase processes, broker handling, and source‑code insights for ensuring data consistency in asynchronous systems.
This article explains why load balancing is crucial for RocketMQ producers, details the internal selection algorithm using ThreadLocal indexes and round‑robin logic, and provides Java code examples illustrating how messages are distributed across queues.
This comprehensive article explains the fundamentals of message‑queue middleware, covering point‑to‑point and publish/subscribe models, key evaluation metrics, detailed comparisons of RabbitMQ, Kafka, RocketMQ and ActiveMQ, and deep dives into Kafka and RocketMQ architectures, features, and best‑practice usage.
This article explains the concept of a local message table, shows how it can be used to achieve reliable distributed transaction consistency with MQ, and provides complete MySQL DDL and Java service code along with a retry mechanism and analysis of failure scenarios.
The author celebrates Programmer's Day by sharing a comprehensive 100,000‑word e‑book on RocketMQ design, detailing personal learning experiences, practical applications, and key concepts to help backend engineers quickly master this powerful message‑queue technology.
This article explains RocketMQ's core components, the differences between push and pull consumption modes, and how the framework implements a pseudo‑push mechanism using long polling, including detailed code analysis of the consumer and broker side processes.
This article explains when adding consumers to a RocketMQ topic improves message consumption, the reasons for consumer pull delays, and details six MessageQueue allocation strategies—including average, round‑robin, custom, machine‑room, nearby, and consistent‑hash—accompanied by code examples.
This article explains the core components of RocketMQ—including NameServer, Broker, Producer, and Consumer—detailing their functions, deployment diagrams, and how the middleware supports high‑concurrency scenarios such as peak shaving, asynchronous messaging, fault tolerance, and system decoupling.
This article explains how Apache RocketMQ performs server‑side message filtering using Tag and SQL92 expressions, outlines the workflow for producers and consumers, compares the two methods, and provides Java code examples for implementing both simple tag filtering and advanced SQL‑based attribute filtering.
This guide walks you through using Alibaba Cloud PTS and JMeter 5.5 to create a custom JavaSampler for RocketMQ 5.0, package the sampler into a JAR, configure a PTS JMeter environment, run an RPS‑mode load test, and analyze the detailed performance report.
This article explains why batch messaging in RocketMQ is crucial for high-throughput systems, outlines its characteristics and ideal scenarios, and provides complete Java code examples for sending and splitting large batches to improve performance and reduce downstream API calls.
This article explains how to apply gray release techniques in RocketMQ by upgrading consumers, upgrading producers, and using consumer‑side filtering, tag filtering, and SQL92 filtering, providing concrete code examples, configuration steps, and trade‑off analysis for each approach.
This article explains seven typical message‑queue use cases—including ordinary, ordered, delayed, transactional, trace, dead‑letter, and priority messages—detailing their business motivations, implementation challenges, and concrete code examples for Kafka, RocketMQ, Pulsar, and RabbitMQ.
This course provides an in‑depth study of RocketMQ, covering its fundamentals, source‑code analysis, advanced features such as transactional and ordered messages, performance optimization, operational monitoring, and hands‑on projects that guide learners through installation, configuration, clustering, and integration with Spring Boot.
This article examines the numerous scheduled tasks in RocketMQ—covering architecture, producer and consumer maintenance, broker housekeeping, and NameServer management—to reveal how periodic jobs ensure reliable message handling, performance monitoring, and system self‑protection.
This article explains the various scheduled tasks in RocketMQ, covering architecture overview, producer and consumer periodic operations, broker maintenance jobs, and NameServer housekeeping, while providing concrete Java code examples to illustrate how each task contributes to message handling, monitoring, and system reliability.
Learn step‑by‑step how to configure Alibaba’s open‑source Canal to capture MySQL binlog changes, route them through RocketMQ, and index the data into Elasticsearch, covering cluster mode, MySQL and Elasticsearch setup, Canal properties, and consumer implementation details.
This tutorial walks through configuring Canal in cluster mode to capture MySQL row‑based binlog events, publish them to a RocketMQ topic, and consume the ordered messages to update an Elasticsearch index, covering MySQL, RocketMQ, Elasticsearch mappings, and the required Canal properties.
This article explains RocketMQ's internal workflow for concurrent and ordered message consumption, covering thread‑pool configuration, pull‑message handling, request submission, execution steps, offset management, expired‑message cleanup, and answers common questions about batch retries and offset ordering.
Explore how Java's CompletableFuture enhances asynchronous programming by overcoming Future's limitations, learn its core APIs, practical code examples, and see a real-world application in RocketMQ where combined futures streamline message persistence and replication.
From the early days of ActiveMQ and RabbitMQ to the high‑throughput designs of Kafka, RocketMQ, and Pulsar, this article traces the development stages of message queues, explains core concepts such as topics, producers, consumers, and partitions, and compares their architectural and storage designs.
