This article explains Kafka's role as a high‑throughput distributed message queue, detailing its core components, topic‑partition model, consumer groups, storage mechanisms, fault‑tolerance features, delivery guarantees, ZooKeeper coordination, and scalability strategies for building reliable real‑time data pipelines.
This article explains the concept, use cases, and common implementations of delay queues—including JDK's DelayQueue, RabbitMQ plugins, and Redis ZSet—followed by a detailed guide to installing, configuring, and using the open‑source lmstfy project, and concludes with promotional information for ChatGPT‑related services.
This article explains the practical problems encountered when using Kafka directly, introduces SFMQ—a Kafka‑based HTTP proxy that unifies client access, simplifies upgrades, provides delay and transaction capabilities via Redis, and demonstrates how its architecture resolves producer, consumer, and operational challenges.
This article explains the concept, working principle, common use cases, and various implementations of delay queues—including JDK's DelayQueue, RabbitMQ plugins, Redis sorted sets, and the Go‑based lmstfy project—along with installation, configuration, and sample client code.
This article explains the motivations for adopting message queues, outlines core concepts and protocols, compares mainstream MQ products, and provides a detailed walkthrough of Kafka architecture, cluster setup, native Java APIs, and Spring Boot integration with extensive code examples.
This article explains query separation as a strategy to accelerate slow large‑table queries by duplicating data to a dedicated query store, outlines when to adopt it, compares synchronous, asynchronous, and binlog approaches, discusses storage choices such as MongoDB, HBase, Elasticsearch, and addresses consistency and MQ challenges.
This article provides a detailed overview of Apache Kafka, covering its core characteristics, distributed architecture, key components such as topics, partitions, brokers, producers, consumers, ZooKeeper, and common application scenarios like log collection, event‑driven architecture, real‑time analytics, and monitoring.
This article provides a comprehensive overview of Apache Kafka, covering its core features, architectural components, message flow, and common application scenarios such as log collection, decoupled messaging, activity tracking, operational monitoring, and stream processing.
This article explains the concept of delayed queues, compares them with scheduled tasks, outlines common business scenarios, and provides a complete Redis implementation using Sorted Sets, Lua scripts, and PHP code, including producer and consumer examples and performance considerations.
This article examines why messages can be lost or duplicated in typical queue systems, explains the failure points from producer to broker to consumer, and provides practical techniques such as synchronous flushing, broker clustering, database unique constraints, and Redis deduplication to achieve reliable, idempotent processing.
This article provides a comprehensive overview of four major message queue systems—Kafka, RabbitMQ, RocketMQ, and ActiveMQ—covering their fundamentals, architectures, key concepts, advantages, disadvantages, and guidance for selecting the most suitable solution based on throughput, reliability, scalability, and use‑case requirements.
This article provides a comprehensive overview of Apache RocketMQ, covering its core concepts, message models, reliability mechanisms, ordered message challenges and solutions, as well as detailed explanations of transactional messages, including implementation principles, producer and broker workflows, and practical considerations for ensuring consistency in distributed systems.
This article explains why message queues are needed, outlines RabbitMQ's architecture, describes its basic concepts and working modes such as simple, work, fanout, direct and topic, and discusses reliability features like transactions, confirms, dead‑letter queues, TTL, clustering, ordering, and handling message backlogs.
This article provides a comprehensive overview of Apache RocketMQ, detailing its core components, producer and consumer workflows, storage strategies, master‑slave synchronization, Raft‑based half‑write and leader election mechanisms, and best‑practice recommendations for high‑throughput, fault‑tolerant messaging systems.
This article provides a detailed introduction to RocketMQ, covering its purpose as a distributed messaging middleware, the roles of NameServer, Producer, Broker, and Consumer, and explains how each component functions within a scalable, high‑throughput architecture.
The article evaluates common delayed‑task implementations—Redis expiration listeners, RabbitMQ dead‑letter queues, time wheels, and Redisson DelayQueue—explaining their drawbacks, proper use cases, and recommending reliable message‑queue solutions for e‑commerce order‑closing scenarios.
This article explores the evolution of message middleware, defines event‑driven architecture, explains why it resurfaces in the cloud era, and details RocketMQ 5.0 EventBridge’s design—its CloudEvents‑based abstraction, schema support, rule engine, observability, and real‑world use cases—offering practical guidance for building scalable, decoupled systems.
This article explains the rapid growth of IoT, the unique messaging requirements it creates, compares classic and IoT message scenarios, and details how RocketMQ 5.0’s MQTT sub‑product introduces a lightweight protocol, read‑amplify storage, edge‑cloud integration, massive queue support, and a push‑pull delivery model to meet cloud‑native IoT demands.
The article systematically compares eleven technical approaches for automatically closing expired e‑commerce orders, detailing each method’s implementation steps, trade‑offs, performance characteristics, and ideal deployment scenarios, and finally ranks them to guide practical selection.
From asynchronous decoupling to distributed transactions, this article explores seven classic message‑queue use cases—such as peak‑shaving, event buses, delayed tasks, broadcast consumption, and data hub integration—illustrated with real‑world examples from e‑commerce, ride‑hailing, and lottery systems, plus RocketMQ code snippets.
The article details how vivo’s Internet Middleware Team migrated from RabbitMQ to RocketMQ and introduced a user‑property‑based multi‑version isolation layer that tags messages, filters them by consumer groups, and provides fallback and retry mechanisms, enabling test‑environment separation without SDK changes, though offline consumers may miss baseline messages.
Message Queues (MQ) are essential for scalable, decoupled systems, offering asynchronous communication, reliability, buffering, and high availability; this guide explains MQ fundamentals, characteristics, architectures like point-to-point and publish/subscribe, core components, design considerations, and reviews popular solutions such as RabbitMQ, Kafka, ActiveMQ, RocketMQ, and Pulsar.
This article introduces the LMAX Disruptor library, explains its core concepts such as RingBuffer, Sequencer and WaitStrategy, and provides a step‑by‑step Java demo—including Maven dependency, model, event factory, handler, manager, service and test code—to build a fast, lock‑free producer‑consumer queue.
This article breaks down the ten key mechanisms—batch sending, message compression, Netty‑based networking, zero‑copy I/O, sequential writes, optimized storage structures, asynchronous flushing and replication, batch processing, lock refinements, and thread‑pool isolation—that together give RocketMQ its remarkable speed and efficiency.
This article walks through the background, core concepts, and a complete hands‑on demo of LMAX Disruptor, showing how to configure Maven, implement ring buffers, event factories, handlers, producers, and a test suite to achieve high‑throughput, lock‑free messaging in Java.
This article compares RabbitMQ and Apache Kafka across architecture, message ordering, routing, timing, retention, fault handling, scalability, and consumer complexity, and provides guidance on which platform suits specific use‑cases such as flexible routing, strict ordering, long‑term retention, or high throughput.
This article introduces the open‑source Disruptor library, explains its core concepts such as Ring Buffer, Sequence, Sequencer and Wait Strategy, and provides a step‑by‑step Java demo—including Maven dependency, event model, handlers, configuration, producer, and test code—to show how to build a fast in‑memory message queue.
This guide walks you through setting up the Java environment, downloading Apache RocketMQ 5.1.4, installing its NameServer, Broker, and optional Proxy components on an Ubuntu 20.04 cloud instance, and configuring both local and cluster deployment modes with optional visual dashboard tools.
This article explains how ByteDance’s BMQ, a cloud‑native message engine with a compute‑storage separated architecture, overcomes Kafka’s scalability and operational limits by using Proxy, Broker, Coordinator, and Controller modules, a distributed storage model, and advanced caching to achieve rapid scaling, high throughput, and resilient operations.
Discover ten practical Redis scenarios—from counting page visits and caching category trees to implementing distributed locks, leaderboards, rate limiting, bitmap analytics, message queues, and global ID generation—each illustrated with concise code snippets and best‑practice tips for robust backend development.
This article provides a comprehensive overview of message queue fundamentals, covering producers, consumers, brokers, point-to-point and publish/subscribe models, ordering, ACK mechanisms, eventual consistency, transactions, persistence, high availability, and selection criteria for various MQ technologies.
This article explains Kafka as a distributed publish/subscribe messaging system, detailing its core functions, architecture, advantages, deployment methods, common use cases, and provides Java consumer and producer code examples for real‑time data processing.
This guide walks through installing RabbitMQ, explaining its features and typical use cases, then details step‑by‑step deployment of a mirrored‑mode RabbitMQ cluster on Kubernetes using StatefulSets, NFS‑backed persistent storage, RBAC, and verification of cluster health and management operations.
This article provides an in‑depth introduction to RocketMQ, covering its evolution, core components, message domain model, key characteristics such as ordered delivery and deduplication, and typical use cases like traffic shaping, asynchronous decoupling, and distributed transaction messaging.
This article explains the fundamentals, architectures, key concepts, and trade‑offs of four popular message‑queue systems—Kafka, RabbitMQ, RocketMQ and ActiveMQ—providing practical guidance for interview preparation and technology selection in modern backend development.
This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.
This article walks you through building a scalable system—from a single‑server prototype to a multi‑data‑center design that uses load balancers, database replication, caching, CDNs, stateless layers, message queues and automated monitoring—to handle traffic from hundreds of thousands to millions of users.
This article introduces message queues, explaining their role in high‑traffic systems, key characteristics such as asynchronous communication, decoupling, reliability and buffering, compares point‑to‑point and publish‑subscribe models, and reviews popular implementations like RabbitMQ, Kafka, ActiveMQ, RocketMQ and Pulsar.
This article explains why message queues can become backlogged, identifies producer and consumer causes, and provides practical strategies—including adding consumers, increasing queue capacity, optimizing consumption logic, implementing failure handling, and rapid remediation steps—to quickly resolve backlog in production environments.
This guide walks you through expanding a single‑server application into a highly available, horizontally‑scaled architecture that can serve over a million users by adding load balancers, database replication, caching layers, CDNs, stateless network design, multi‑data‑center support, message queues, monitoring and automation.
This article compares eight ways to implement retry logic for remote API calls—including simple loops, recursion, Apache HttpClient settings, Spring Retry, Resilience4j, a custom utility, thread‑pool asynchronous retries, and message‑queue based retries—while outlining best‑practice guidelines such as idempotency, retry limits, and concurrency control.
This article explains Kafka's fundamental components, version evolution, key monitoring metrics for producers, brokers, consumers and Zookeeper, and provides step‑by‑step troubleshooting methods for common issues such as slow topic throughput and message backlog.
This article introduces five widely used message middleware systems—Kafka, RabbitMQ, ActiveMQ, RocketMQ, and Pulsar—explaining their core architectures, key components, typical use cases, and advantages for building reliable, high‑throughput backend solutions.
Kafka, an Apache‑developed distributed publish/subscribe messaging system, provides reliable, high‑throughput real‑time data streaming with producers, consumers, brokers, streams, and connectors, and the article explains its core concepts, architecture, advantages, deployment methods, use cases, and includes Java code examples for producers and consumers.
This article walks through the full design of a billion‑user group chat system, covering functional and non‑functional requirements, core components such as WebSocket and Redis, face‑to‑face group creation, message flow, database schema, and performance‑oriented scaling techniques.
This article explains why MQ is needed beyond in‑process event listeners, details the required dependencies for Spring Boot 3, shows how to configure RocketMQ, provides code snippets for producer, consumer, and testing, and offers practical tips for handling async, batch, and one‑way messaging.
This article explores Redis's capabilities beyond simple caching, detailing its use for distributed locks, rate limiting, session storage, complex business scenarios, and various messaging patterns—including List‑based queues, blocking commands, Pub/Sub, and the Stream data structure—while highlighting practical limitations and best‑practice recommendations.
This article explains why introducing an MQ middleware helps with system decoupling and traffic control, outlines the data‑consistency challenges it creates, and provides practical methods to detect lost messages, identify loss points in producer, broker, and consumer stages, and configure Kafka to guarantee reliable delivery.
This article walks through installing RabbitMQ 3.12.6‑management via Docker‑Compose V2, covering image selection, configuration details, plugin setup for delayed messages, common pitfalls, and .NET client connection examples, providing practical tips for reliable messaging in backend systems.
This article records a complete, step‑by‑step installation and startup process for Apache RocketMQ on a CentOS 7 system, covering environment checks, downloading the package, extracting files, launching the name server and broker, adjusting memory settings, and verifying the setup with a test client.
This article walks through the full design of a large‑scale comment system, covering functional requirements, database schema, distributed architecture, high‑concurrency handling, caching, security measures like Bloom filters, and optional NLP‑based sentiment analysis to moderate live‑stream comments.
This guide explains RabbitMQ fundamentals—including brokers, virtual hosts, connections, channels, exchanges, queues, producers, and consumers—covers common messaging patterns, provides step‑by‑step Java code for creating connections, declaring exchanges and queues, publishing and consuming messages, handling acknowledgments and confirmations, and shows how to integrate RabbitMQ with Spring Boot.
This article examines seven typical messaging scenarios—ordinary, ordered, delayed, transactional, trace, dead‑letter, and priority messages—explaining the problem each solves, the trade‑offs, and concrete implementations in RocketMQ, Pulsar, RabbitMQ and Kafka with code samples.
This article explores how Kafka serves as a versatile backbone for messaging, durable storage, log aggregation, monitoring, commit logs, recommendation pipelines, stream processing, CDC, system migration, and event sourcing, highlighting its performance, reliability, and practical deployment patterns.
This article describes the background, problems, and a series of architectural upgrades—including MQ replacement, thread‑pool isolation, Redis/TiKV redundancy, and Spark‑based compensation—to enhance the stability, scalability, and high‑availability of an advertising billing system.
This article introduces the high‑performance Java library Disruptor, explains its core concepts such as Ring Buffer, Sequence, and Wait Strategy, and provides a step‑by‑step demo with Maven configuration, code examples, and test results to help developers implement fast, lock‑free producer‑consumer queues.
This article explains the principles of distributed transaction messages, comparing 2PC, TCC, and transactional messaging, and provides detailed walkthroughs of RocketMQ and Kafka implementations, including their two‑phase processes, broker handling, and source‑code insights for ensuring data consistency in asynchronous systems.
This article introduces the open‑source Disruptor framework, explains its core concepts such as Ring Buffer, Sequencer, and Wait Strategies, and provides a step‑by‑step Java demo—including Maven setup, event factory, handlers, and a test case—to illustrate building a high‑throughput, low‑latency in‑memory message queue.
This article explains why load balancing is crucial for RocketMQ producers, details the internal selection algorithm using ThreadLocal indexes and round‑robin logic, and provides Java code examples illustrating how messages are distributed across queues.
The author celebrates Programmer's Day by sharing a comprehensive 100,000‑word e‑book on RocketMQ design, detailing personal learning experiences, practical applications, and key concepts to help backend engineers quickly master this powerful message‑queue technology.
This article explains Kafka's core components and features, then details seven practical scenarios—including log processing, recommendation streams, monitoring, CDC, system migration, event sourcing, and message queuing—showing how Kafka powers modern distributed systems.
This article explains RocketMQ's core components, the differences between push and pull consumption modes, and how the framework implements a pseudo‑push mechanism using long polling, including detailed code analysis of the consumer and broker side processes.
This article introduces Apache Kafka’s core components and key features, then details seven practical use cases—including log processing, recommendation streams, monitoring, CDC, system migration, event sourcing, and message queuing—illustrated with diagrams and step‑by‑step workflows for distributed systems.
This article explores eight practical retry mechanisms for handling unreliable third‑party APIs in backend systems, covering simple loops, recursion, built‑in client retries, Spring Retry, Resilience4j, custom utilities, asynchronous thread‑pool retries, and message‑queue based approaches, plus best‑practice tips.
This article recounts the step‑by‑step evolution of a ride‑hailing order service’s data layer—from a single SQLServer database through SQL optimization, read‑write splitting, business‑domain sharding, caching and MQ integration, to a custom sharding component and smooth migration to MySQL—highlighting challenges, solutions, and lessons learned.
The article explains how Redis’s rich data structures and persistence options enable it to serve not only as a distributed cache but also as a primary store for user profiles, counters, leaderboards, relationships, active‑user tracking, distributed locks, rate limiting, messaging, geospatial queries, and more, while also discussing high‑availability and hot‑cold data challenges.
This article explains Kafka's fundamentals, including topics, partitions, brokers, replication, producer‑consumer workflow, consumer groups, offset management, and common exception handling, while providing code examples and diagrams to help developers understand and effectively use this distributed messaging system.
This article introduces the LMAX Disruptor library, explains its core concepts such as Ring Buffer, Sequencer and Wait Strategy, and provides a step‑by‑step Java demo with full source code to build a fast, lock‑free in‑memory message queue.
This article explains the core components of RocketMQ—including NameServer, Broker, Producer, and Consumer—detailing their functions, deployment diagrams, and how the middleware supports high‑concurrency scenarios such as peak shaving, asynchronous messaging, fault tolerance, and system decoupling.
This article explains why batch messaging in RocketMQ is crucial for high-throughput systems, outlines its characteristics and ideal scenarios, and provides complete Java code examples for sending and splitting large batches to improve performance and reduce downstream API calls.
This article introduces asynchronous messaging patterns, then compares RabbitMQ and Apache Kafka by examining their internal architectures, message models, and trade‑offs, helping architects choose the appropriate solution based on scenario requirements in modern.
This article explains how to apply gray release techniques in RocketMQ by upgrading consumers, upgrading producers, and using consumer‑side filtering, tag filtering, and SQL92 filtering, providing concrete code examples, configuration steps, and trade‑off analysis for each approach.
This article explains seven typical message‑queue use cases—including ordinary, ordered, delayed, transactional, trace, dead‑letter, and priority messages—detailing their business motivations, implementation challenges, and concrete code examples for Kafka, RocketMQ, Pulsar, and RabbitMQ.
This article presents a complete Netty TCP client demo built with Spring Boot, detailing the project architecture, module layout, business flow, key code implementations—including a local BlockingQueue, message processing with Redis locks, client initialization, handler logic, and testing endpoints—providing a practical reference for backend developers.
This article compares RabbitMQ and Apache Kafka across dimensions such as message ordering, routing, timing, retention, fault handling, scalability, and consumer complexity, and provides practical guidance on when to choose each platform for backend architectures.
This article explains the fundamental asynchronous messaging patterns, compares RabbitMQ and Apache Kafka in terms of architecture, queue and pub/sub models, and provides guidance on selecting the appropriate technology for different backend scenarios.
This article explains Kafka’s fundamental role as a message system, detailing topics, partitions, producers, consumers, replica management, consumer groups, the controller, Zookeeper coordination, and performance optimizations such as sequential writes, zero‑copy, log segmentation, and network design, providing a comprehensive overview for big‑data practitioners.
The article recounts a series of real‑world anecdotes exposing how excessive architectural ambitions—such as needless microservice fragmentation, ubiquitous message‑queue usage, multi‑level caching, over‑engineered design patterns, and configuration‑driven code—lead to wasted resources, maintenance headaches, and fragile systems, urging a return to simplicity.
This course provides an in‑depth study of RocketMQ, covering its fundamentals, source‑code analysis, advanced features such as transactional and ordered messages, performance optimization, operational monitoring, and hands‑on projects that guide learners through installation, configuration, clustering, and integration with Spring Boot.
This article explains Kafka's role as a messaging system, covering core concepts such as topics, partitions, producers, consumers, messages, cluster architecture, replicas, consumer groups, controller coordination with Zookeeper, and performance optimizations like sequential writes and zero‑copy networking.
This article details the architecture, technical requirements, and implementation techniques—including hybrid push‑pull, heterogeneous storage, distributed scheduling, task chunking, and at‑least‑once delivery—used to build a high‑concurrency, reliable reminder notification service for QQ.
This guide explains how to implement delayed and priority queues in RabbitMQ 3.8 using Spring Boot 2.6, covering dead‑letter exchange configuration, message expiration, priority settings, code examples for sending and consuming messages, and practical considerations such as priority limits and ordering behavior.
This tutorial walks through RabbitMQ fundamentals, installation on macOS, core AMQP concepts, common exchange types, and step‑by‑step SpringBoot integration with code samples, while highlighting practical issues like connection leaks and the need for a connection pool.
This article provides an in‑depth technical comparison of RabbitMQ and Apache Kafka, covering their core architectural differences, message ordering, routing, timing, retention, fault handling, scalability, consumer complexity, and offers guidance on selecting the appropriate platform for various backend scenarios.
This article explains how to detect, diagnose, and prevent message loss in Kafka-based message queue systems by covering system decoupling, traffic control, data consistency challenges, producer, broker, and consumer issues, and offering configuration, monitoring, and operational best‑practice solutions.
This article explains RocketMQ's internal workflow for concurrent and ordered message consumption, covering thread‑pool configuration, pull‑message handling, request submission, execution steps, offset management, expired‑message cleanup, and answers common questions about batch retries and offset ordering.
This article explains why using Redis List for a message queue is limited, introduces Redis Stream as a purpose‑built MQ with consumer groups, ACKs, and persistence, and dives into its internal Radix‑Tree and listpack structures, command usage, and memory‑saving techniques.
This article introduces Apache Pulsar, an open‑source cloud‑native distributed messaging platform, detailing its storage‑compute separation architecture, multi‑tenant support, load balancing, fault tolerance, schema handling, functions, IO connectors, tiered storage, cross‑region replication, and operational commands for managing brokers, bookies, and namespaces.
This article breaks down real‑world high‑concurrency strategies—horizontal scaling, caching, Elasticsearch, sharding, message‑queue smoothing, and cellization—explaining when each applies, their trade‑offs, and practical tips for building scalable, reliable backend services.
This article explains how ordering issues in message queues like RabbitMQ and Kafka can break MySQL binlog synchronization, illustrates common pitfalls, and offers practical solutions to guarantee correct processing order in high‑throughput backend systems.
This article provides a detailed overview of popular message queue systems, explains their core concepts such as decoupling and eventual consistency, and compares the advantages and disadvantages of Kafka, RocketMQ, RabbitMQ, and other notable MQ solutions for high‑concurrency scenarios.
Redis is a high-performance, open-source in-memory key-value database offering rich data structures, persistence options, and advanced features like replication, clustering, and pub/sub, making it ideal for caching, counters, message queues, leaderboards, and session storage in demanding, high-concurrency applications.
The article analyzes the root causes and system‑wide consequences of oversized messages in JD Logistics, explains middleware limits of JMQ and JSF, and presents design principles, code‑level checks, and practical mitigation techniques such as pagination, claim‑check pattern, batch sizing, and monitoring to prevent service outages.
This article compares Kafka and RabbitMQ across scalability, durability, latency, data flow, ordering, reliability, persistence, extensibility, and complexity, then outlines ideal use cases for each and offers practical guidance on selecting the right message queue for a project.
This article explains the three core benefits of message queues, compares major MQ products, details RocketMQ's advantages, internal models, reliability mechanisms, load‑balancing, storage design, and practical code examples, guiding engineers to make informed architectural decisions.
This guide explains what a process is, how it differs from a thread, various ways to create and terminate processes in Linux, and provides detailed examples of inter‑process communication mechanisms—including message queues, shared memory, UNIX domain sockets, pipes, and semaphores—complete with C code samples.
This article provides a thorough overview of message queues, covering their definition, core components, types (point‑to‑point and publish/subscribe), key characteristics such as asynchrony, loose coupling, distribution and reliability, selection criteria among popular MQ products, and typical application scenarios in high‑concurrency systems.
This article details the architectural redesign of QQ Music's comment platform—migrating to MongoDB, introducing threaded comments, and employing caching and message‑queue decoupling—to handle massive read/write spikes during celebrity live‑drop events while maintaining high availability and performance.
QQ Music’s comment system was re‑engineered with a MongoDB backend, cache layer, asynchronous writes, split read/write services, priority queues and rate‑limiting, enabling it to endure celebrity‑driven traffic spikes, maintain data consistency, and deliver high‑availability, low‑latency user experiences.
This tutorial explains how to create delayed and priority queues in RabbitMQ using Spring Boot, covering dead‑letter queue configuration, TTL settings, priority parameters, and provides complete Java code examples for sending and consuming prioritized messages.
