Tagged articles
Distributed Messaging
72 articles · Page 1 of 1
This article provides a comprehensive overview of RocketMQ, covering its core components (Producer, Broker, Consumer, NameServer), detailed architecture diagrams, step‑by‑step message production and consumption processes, and typical scenarios such as asynchronous decoupling, traffic shaping, transactional messaging, and event‑driven logging.
ConsumerDistributed MessagingMessage Queue
0 likes · 6 min read

This article explains Kafka’s core architecture—producer, broker storage, and consumer groups—detailing how messages are partitioned, buffered, replicated, and consumed to achieve high throughput, low latency, and scalable real‑time data processing.
Distributed MessagingKafkaarchitecture
0 likes · 4 min read

This article provides a comprehensive overview of Kafka, covering its high‑throughput distributed messaging model, key components such as producers, consumers, brokers, topics, partitions, and metadata management, as well as the end‑to‑end workflow from production to consumption.
Distributed MessagingKafkastreaming
0 likes · 5 min read

The article explains RocketMQ NameServer's lightweight, stateless design, its core routing and metadata management functions, AP‑oriented architecture, fault‑tolerant mechanisms, scalability features, and practical optimization techniques for high availability and low operational cost.
Distributed MessagingNameServerRocketMQ
0 likes · 6 min read

This guide walks you through four progressive stages—basic configuration, SSL encryption, SCRAM authentication, and combined SSL+SASL—showing how to harden Kafka with certificates, keystores, and client settings to achieve financial‑level protection.
Distributed MessagingKafkaSASL
0 likes · 9 min read

This article explains how RocketMQ tackles synchronous registration bottlenecks, tight coupling, and traffic‑burst risks by introducing an intermediate queue layer, designing a durable high‑availability broker, leveraging page cache and mmap for zero‑copy I/O, and using a nameserver for automatic routing, ultimately delivering a high‑throughput, low‑latency messaging system.
Distributed MessagingRocketMQbroker design
0 likes · 27 min read

This article explains RocketMQ’s fundamental architecture, detailing the definitions, roles, and interactions of topics, message queues, messages, producers, consumers, consumer groups, and subscription relationships, and highlights how these components work together to achieve high performance, availability, and scalability.
Distributed MessagingMessage QueueMessaging Architecture
0 likes · 7 min read

This article dissects RocketMQ's concurrency mechanisms, explaining why it uses ReentrantReadWriteLock with HashMaps, how Semaphore controls async send rates, the SemaphoreReleaseOnlyOnce wrapper, the CountDownLatch‑based sync‑to‑async pattern, and the CompletableFuture redesign that reduces blocking in message replication.
CompletableFutureConcurrencyDistributed Messaging
0 likes · 10 min read

When a SMS service fails and messages need to be re‑sent, this article shows how to use RocketMQ's message backtracking feature to reset consumer offsets, verify the reset, and explore the internal workflow and alternative pull‑consumer strategies with concrete code examples and detailed analysis.
Distributed MessagingJavaMessage Backtracking
0 likes · 21 min read

This article shares the architecture design, cluster deployment, storage selection, multi‑region mode, service discovery, performance tuning, monitoring, alerting, and future plans of a production‑grade Apache Pulsar platform running on Kubernetes, providing valuable insights for engineers adopting Pulsar.
Apache PulsarDistributed MessagingKubernetes
0 likes · 17 min read

This article shares 360's practical experience deploying Apache Pulsar on Kubernetes, covering architecture design, storage choices, multi‑cluster modes, service discovery, performance tuning, monitoring, alerting, and future plans, offering valuable guidance for engineers building cloud‑native messaging platforms.
Apache PulsarDistributed MessagingKubernetes
0 likes · 19 min read

This technical presentation details Vivo's end‑to‑end big‑data architecture, the evolution from Kafka to Pulsar for massive message processing, deployment strategies, high‑availability mechanisms, observability practices, and future plans for cloud‑native, containerized messaging middleware.
Distributed MessagingHigh AvailabilityKafka
0 likes · 18 min read

This article provides a comprehensive guide to Apache Kafka, covering its typical use cases, fundamental terminology, step‑by‑step installation and environment preparation, detailed configuration of brokers and topics, cluster deployment, producer‑consumer commands, and a Java client example for sending messages.
Distributed MessagingJava ClientProducer-Consumer
0 likes · 21 min read

This article explains how Apache Pulsar's GEO Replication synchronizes subscription state between primary and backup clusters, detailing the challenges of ledger mismatches, the two‑step mapping process for MessageIds, and the mechanisms that ensure seamless failover without duplicate consumption.
Apache PulsarDistributed MessagingGEO Replication
0 likes · 8 min read

This article explains how to enable cross‑node WebSocket communication in a distributed Spring application by publishing messages to a Redis or Kafka topic, tracking user connections, and routing messages to the appropriate server instance, complete with full code examples and configuration details.
Distributed MessagingKafkaSpring
0 likes · 16 min read

This article introduces Kafka, a high‑performance distributed messaging system, and provides step‑by‑step Java/Groovy examples for configuring a producer and consumer, demonstrating how to benchmark their throughput using the FunQpsConcurrent framework, along with necessary Gradle dependencies and server setup instructions.
ConsumerDistributed MessagingJava
0 likes · 8 min read

This comprehensive guide explores Kafka as a distributed messaging middleware, detailing its core concepts, architecture, producer and consumer mechanisms, configuration options, Zookeeper integration, controller responsibilities, network model, performance optimizations such as zero‑copy, page‑cache usage, batching, compression, and partition concurrency.
Distributed MessagingKafkaZooKeeper
0 likes · 41 min read

This guide outlines Kafka stability best practices across three phases—pre‑prevention with tuning, producer/consumer guidelines, and cluster configuration; runtime monitoring using white‑box and black‑box metrics and alerts; and fault resolution strategies for backlogs, consumption blocks, and message loss, plus cost control and idempotence techniques.
Distributed MessagingKafkaPerformance Tuning
0 likes · 29 min read

This article explains Kafka's distributed publish‑subscribe architecture, detailing its core components, underlying mechanisms with Zookeeper coordination, key features such as high throughput and fault tolerance, and common application scenarios like log collection, user activity tracking, and stream processing.
Distributed MessagingKafkaZooKeeper
0 likes · 5 min read

This article provides a comprehensive overview of Apache Kafka, explaining its distributed message‑queue architecture, the role of topics and partitions, producer and consumer workflows, leader election, offset management, consumer‑group rebalancing, delivery semantics, transaction processing, file organization, and key configuration settings.
Distributed MessagingKafkaTransactions
0 likes · 17 min read

Kafka is a high‑performance distributed message queue that uses topics, partitions, and replicas for scalability, with producers writing messages, consumers reading them via consumer groups, and advanced features such as offset management, delivery semantics, rebalancing, and exactly‑once transactional processing, all coordinated by Zookeeper.
ConsumersDistributed MessagingProducers
0 likes · 17 min read

This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, replication, producer and consumer workflows, offset management, rebalancing, delivery semantics, exactly‑once guarantees, transaction handling, file organization, and key configuration settings.
Distributed MessagingReplicationconsumer groups
0 likes · 17 min read

This comprehensive guide explains Kafka’s origins, core architecture, data structures, write and read workflows, operational commands for topic and consumer‑group management, and practical performance‑tuning tips such as disk layout, JVM settings, flush policies, and log retention, providing a complete reference for engineers working with distributed streaming platforms.
Data StreamingDistributed MessagingKafka
0 likes · 32 min read

This article explains how to solve cross‑server WebSocket communication in a distributed Java application by using a message queue (Redis or Kafka) to broadcast messages, storing user‑node mappings, and providing complete Spring configuration, code examples, and a demo page for end‑to‑end testing.
BackendDistributed MessagingJava
0 likes · 18 min read

This article provides an in‑depth overview of Kafka, covering why messaging systems are needed, core concepts, cluster architecture, performance optimizations such as sequential disk writes and zero‑copy, hardware sizing, replication, consumer groups, offset management, rebalance strategies, and practical deployment and operational guidelines.
Cluster DeploymentDistributed MessagingKafka
0 likes · 35 min read

This article provides a comprehensive overview of Apache Pulsar, detailing its multi‑tenant architecture, core components such as brokers and BookKeeper, message handling features, producer and consumer configurations, comparison with Kafka, and operational considerations for deployment.
Apache PulsarBookKeeperConsumers
0 likes · 21 min read

Apache Pulsar is a cloud‑native distributed messaging system that combines stateless brokers, a BookKeeper storage cluster, and ZooKeeper metadata to route messages from producers to consumers across scalable partitioned or non‑partitioned topics, supporting multiple subscription models, producer access modes, routing strategies, batching, acknowledgments, delayed delivery, and deduplication.
Apache PulsarBookKeeperConsumer
0 likes · 26 min read

This article provides a comprehensive overview of Apache Kafka, covering its distributed architecture, key advantages and drawbacks, the role of ZooKeeper, message delivery semantics, partitioning strategies, storage mechanisms, and performance optimizations such as zero‑copy and batch processing, all essential for high‑throughput real‑time data pipelines.
Distributed Messagingbig datastreaming
0 likes · 23 min read

Apache Pulsar ensures reliable delivery by letting consumers acknowledge messages individually, cumulatively, or within batches, while also supporting negative acknowledgments and sophisticated tracking mechanisms that batch ack requests, use BitSet for batch indexes, and employ time‑wheel‑based redelivery for unacknowledged messages.
Apache PulsarConsumer TrackerDistributed Messaging
0 likes · 10 min read

This article presents a detailed overview of Alipay's Android wallet system architecture, illustrating its core components such as settlement, customer service, fund processing, and accounting, and further examines the open‑source MetaQ (Metamorphosis) distributed messaging middleware, highlighting its features, advantages over Kafka, and suitable application scenarios.
AlipayAndroidBackend
0 likes · 5 min read

This article provides a comprehensive technical overview of Apache Kafka, covering its distributed messaging architecture, key features such as high‑throughput read/write, replication, partitioning, consumer group mechanics, offset management, rebalance processes, and practical code examples for synchronous and asynchronous offset commits.
Consumer OffsetsDistributed MessagingKafka
0 likes · 22 min read

Apache Pulsar is a cloud‑native distributed messaging platform that combines messaging, storage, and lightweight compute, featuring multi‑tenant support, geo‑replication, and high throughput, and this article introduces its core concepts, architecture components such as brokers, BookKeeper, ZooKeeper, and key design features.
Apache PulsarBookKeeperDistributed Messaging
0 likes · 13 min read

This article explains Apache RocketMQ's high‑performance distributed messaging architecture, detailing its core components—Producer, Broker, Consumer, and Namesrv—along with routing metadata, message ordering, priority handling, parallelism, and essential concepts such as topics, groups, tags, and reliability mechanisms.
Apache RocketMQDistributed MessagingMessage Queue
0 likes · 14 min read

This article details how Lakala built a distributed, cloud‑native messaging platform using Apache Pulsar, covering functional requirements, architectural advantages, performance testing, and real‑world integration scenarios such as OGG adapters, TiDB pipelines, OpenMessaging, custom sources, functions, Flink connectors, and future plans.
Apache PulsarDistributed Messagingbackend architecture
0 likes · 18 min read

Kafka is a high‑throughput distributed publish‑subscribe system that uses brokers, topics, partitions, offsets, producers, consumers, and Zookeeper for metadata and leader election, offering fast sequential disk writes, page‑cache zero‑copy transfers, ISR‑based replication, and includes step‑by‑step installation of JDK, Zookeeper, and Kafka.
Distributed MessagingKafkaZooKeeper
0 likes · 11 min read

This article provides a comprehensive overview of Kafka, covering its role as a distributed messaging middleware, core concepts, architecture components, common interview questions, command‑line tools, producer and consumer configurations, high‑availability mechanisms, delivery semantics, and performance optimizations for backend developers.
ConsumerDistributed MessagingKafka
0 likes · 20 min read

Apache Pulsar’s built‑in Geo‑Replication lets multiple clusters across different regions synchronize data, offering both synchronous and asynchronous modes; this guide explains the three asynchronous patterns—full‑mesh, unidirectional, and failover—detailing required configurations, operational principles, and current limitations.
Apache PulsarCluster ConfigurationDistributed Messaging
0 likes · 10 min read

This comprehensive guide introduces Kafka as a high‑throughput, distributed, publish‑subscribe messaging system, detailing its core concepts, architecture, features, replication, log management, reliability guarantees, and typical use cases such as log collection, real‑time analytics, and cross‑cluster mirroring.
Distributed MessagingKafkabig data
0 likes · 15 min read

This article explains RocketMQ’s core features, the operational challenges of using its console, and how the MQCloud platform was designed to separate business and admin roles, provide comprehensive monitoring, automated deployment, security hardening, and a customized client, ultimately turning operational pain into a scalable, open‑source solution.
Distributed MessagingMQCloudRocketMQ
0 likes · 15 min read

The Apache Pulsar Go client, a Go‑language SDK for the cloud‑native distributed messaging platform, has been updated to version 0.3.0, adding Key Shared support, schema capabilities for producers and consumers, and configurable reconnection retry attempts.
Apache PulsarBackendDistributed Messaging
0 likes · 2 min read

This article presents a detailed overview of Alipay's multi‑layer system architecture, explains the design and capabilities of its open‑source Java‑based distributed message middleware Metamorphosis (MetaQ), and discusses why it was created as an alternative to Kafka and the scenarios where it excels.
AlipayDistributed MessagingJava
0 likes · 6 min read

This article provides a comprehensive overview of Kafka as a distributed messaging middleware, covering its core concepts, architecture, producer and consumer mechanics, common interview questions, configuration options, high‑availability guarantees, and performance optimizations for backend developers.
ConsumerDistributed MessagingKafka
0 likes · 20 min read


21CTOOct 20, 2020 · Backend Development
The article examines the scaling challenges of Cainiao's logistics platform during Alibaba's Double‑11 shopping festivals and explains how adopting RocketMQ for asynchronous decoupling dramatically reduced costs, stabilized the transaction‑logistics chain, and enabled sustainable high‑throughput processing.
AlibabaDistributed MessagingDouble 11
0 likes · 6 min read


ITPUBJul 10, 2020 · Backend Development
Facing soaring order peaks each Double‑11, Alibaba's logistics arm re‑engineered its architecture with RocketMQ, using asynchronous decoupling to handle payment‑order traffic, dramatically reduce hardware spend, and keep logistics QPS stable without sacrificing user experience.
AlibabaDistributed MessagingRocketMQ
0 likes · 7 min read


21CTOJun 13, 2020 · Backend Development
This article examines the logistical challenges faced by Alibaba's Cainiao platform during Double‑11, explains why traditional databases and Redis fall short, and shows how adopting RocketMQ for asynchronous decoupling dramatically reduced costs and stabilized high‑volume order processing.
Distributed MessagingRocketMQbackend architecture
0 likes · 6 min read

This article explains Kafka's core architecture, covering how topics and partitions are stored, the advantages of its consumption model, the internal network and threading design, and the high‑reliability distributed log storage and replication mechanisms that ensure data durability and scalability.
Consumer GroupDistributed MessagingHigh Availability
0 likes · 11 min read

This article presents an overview of Alipay’s comprehensive system architecture and introduces MetaQ (Metamorphosis), an open‑source Java‑based distributed messaging middleware, detailing its features, advantages over Kafka, and suitable high‑throughput, reliable messaging scenarios.
AlipayBackendDistributed Messaging
0 likes · 5 min read

This article provides an in‑depth yet approachable overview of Kafka's core concepts—including its architecture, terminology, message‑sending pipeline, replication strategy, controller role, and consumer group rebalance mechanisms—helping readers quickly grasp how Kafka works as a high‑throughput distributed messaging and streaming platform.
Consumer RebalanceDistributed MessagingKafka
0 likes · 21 min read

Xiaomi’s Talos, a self‑developed distributed message queue, tackles the limitations of Kafka by separating storage and compute on HDFS, introducing stateless scaling, advanced consistency, partition delay allocation, and extensive performance and resource optimizations to support trillions of daily messages and multi‑tenant workloads.
Distributed MessagingStorage-Compute SeparationTalos
0 likes · 16 min read


ITPUBDec 2, 2019 · Backend Development
This article details Xiaomi's journey from Kafka 0.8 to the home‑grown Talos system, covering business motivations, storage‑compute separation architecture, key challenges such as tail‑read and consistency, and extensive performance, resource, and platform optimizations that enable a high‑throughput, multi‑tenant messaging service.
Consistent HashingDistributed MessagingHDFS
0 likes · 16 min read

This article provides a comprehensive overview of Kafka, covering its core features, use‑case scenarios, partition and replica design, file storage structure, consumer‑group coordination, delivery guarantees, performance optimizations, and the role of Zookeeper in managing the cluster.
Distributed MessagingKafkaReplication
0 likes · 54 min read

This article introduces Kafka, covering its definition, core concepts such as topics, partitions, offsets, producers and consumers, typical use cases, underlying design principles including message‑partition allocation and retention policies, processing mechanisms, and common troubleshooting questions for real‑world deployments.
Distributed MessagingKafkabig data
0 likes · 7 min read

This article provides an overview of Apache Pulsar, detailing its enterprise‑grade features, architectural components such as brokers, BookKeeper and ZooKeeper, core concepts like topics and metadata, and the supported consumption modes, illustrating why it is considered a next‑generation distributed messaging system.
Apache PulsarBackendDistributed Messaging
0 likes · 9 min read

This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, producer and consumer workflows, replication and leader election, offset management, consumer group coordination, rebalance processes, delivery semantics (at‑most‑once, at‑least‑once, exactly‑once), transactional messaging, and underlying file and configuration details.
Distributed MessagingKafkaReplication
0 likes · 16 min read

Didi Mobility has open‑sourced DDMQ, an enterprise‑grade distributed message queue that delivers millisecond‑level latency and millions of messages per second, supports real‑time, delayed and transactional messages via RocketMQ/Kafka and RocksDB, offers multi‑language SDKs, flexible consumption, server‑side filtering, a unified console, and achieves five‑nine availability across Didi’s ride‑hailing, finance and delivery platforms.
DDMQDistributed MessagingMessage Queue
0 likes · 7 min read

This article introduces Apache RocketMQ by covering its origin, core concepts such as topics, producers and consumers, storage architecture with CommitLog and ConsumeQueue, deployment components like brokers and name servers, and practical best‑practice guidance for handling duplicates, ordering, and message replay.
ApacheBest PracticesDistributed Messaging
0 likes · 8 min read

This article presents a comprehensive overview of Alipay’s system architecture, detailing its core components such as payment processing, accounting, settlement, and transaction modules, and introduces the open‑source Java‑based distributed messaging middleware Metamorphosis (MetaQ), highlighting its features, advantages over Kafka, and suitable use cases.
AlipayDistributed MessagingJava
0 likes · 6 min read

This article provides a comprehensive overview of Apache Kafka, comparing it with ActiveMQ, explaining its distributed architecture, topics, partitions, consumption models, high‑availability mechanisms, exactly‑once semantics, and includes detailed Java producer and consumer code examples for practical implementation.
ConsumerDistributed MessagingKafka
0 likes · 22 min read

This article explains Kafka's origins, design goals, core architecture, message routing, consumer groups, push‑vs‑pull semantics, and delivery guarantees, while comparing it with other popular message‑queue systems to show why it is widely adopted in modern data pipelines.
Delivery GuaranteesDistributed Messagingarchitecture
0 likes · 22 min read

The OpenMessaging project, launched by Alibaba with partners like Yahoo and Didi, aims to create a vendor‑neutral, cloud‑native messaging standard, detailing its value, current implementation in RocketMQ, and a three‑year roadmap covering open‑source releases, cloud platform integration, and ecosystem expansion.
Apache RocketMQDistributed MessagingOpenMessaging
0 likes · 7 min read

This article introduces Apache Kafka as a fast, scalable distributed publish‑subscribe system, explains its core components, Zookeeper coordination, startup workflow, key features, and common scenarios such as log collection, activity tracking, and stream processing.
Distributed MessagingZooKeeperarchitecture
0 likes · 7 min read

The article explains how to build a highly scalable large‑website architecture by adopting modularization, reducing coupling through distributed message queues and services, and exposing functionality via an open platform, covering core concepts, typical implementations like ActiveMQ and Dubbo, and essential design considerations.
Distributed MessagingOpen Platformlarge-scale web
0 likes · 8 min read


21CTOJun 11, 2017 · Big Data
This article explains Kafka's distributed architecture, topic‑partition model, replication and ISR mechanisms, data durability settings, delivery guarantees, deduplication strategies, and presents benchmark results that illustrate how configuration choices affect throughput and latency in real‑world deployments.
Distributed MessagingHigh reliabilityKafka
0 likes · 33 min read

This article explains Kafka’s core architecture—including brokers, topics, partitions, offsets, producer and consumer mechanics, replication, availability, consistency, persistence, performance optimizations, and Zookeeper integration—providing a comprehensive guide for building reliable distributed messaging systems.
Distributed MessagingKafkaOFFSET
0 likes · 15 min read

Tencent Cloud CMQ is a Raft‑based distributed message queue that delivers high reliability, strong consistency, and horizontal scalability for finance‑grade workloads, using multi‑node broker sets with majority‑acknowledged writes, automatic leader election, unlimited buffering, full‑path tracing, while requiring application‑level idempotency and offering limited strict ordering.
CMQDistributed MessagingRaft
0 likes · 11 min read

This article recounts Alibaba's RocketMQ journey—from record‑breaking low‑latency performance during Double‑Eleven, through the challenges of open‑source donation, to its successful incubation at Apache and future plans for both community and commercial versions.
ApacheDistributed MessagingRocketMQ
0 likes · 14 min read

This article presents a detailed overview of Alipay's system architecture—including its processing flow, fund handling platform, accounting, settlement, and transaction modules—and introduces Metamorphosis, an open‑source high‑performance distributed messaging middleware developed by Alibaba, highlighting its features, design motivations, and suitable use cases.
AlipayBackendDistributed Messaging
0 likes · 6 min read

This article explains how Kafka’s design—using sequential disk reads/writes, zero‑copy system calls, file segmentation, batch sending, and message compression—delivers massive throughput while minimizing performance loss and network load.
Data CompressionDistributed Messagingbatch sending
0 likes · 4 min read


21CTOFeb 23, 2016 · Big Data
Kafka, the open‑source distributed messaging system from LinkedIn, offers O(1) persistence, high throughput, partitioned topics, and flexible delivery guarantees, making it a cornerstone for modern big‑data pipelines and real‑time processing alongside Hadoop, Spark, and Storm.
ConsumerDelivery GuaranteesDistributed Messaging
0 likes · 21 min read

This article explains Apache Kafka’s core concepts, high‑throughput design choices such as sequential I/O, PageCache, Sendfile, and partitioning, and provides practical performance tips and configuration recommendations for brokers, producers, and consumers in large‑scale data pipelines.
ConsumerDistributed MessagingKafka
0 likes · 16 min read


21CTONov 21, 2015 · Big Data
This article explains why Kafka is essential for activity and operational data pipelines, outlines key use cases such as news feeds, relevance ranking, security, monitoring, and reporting, and details its deployment topology, design decisions, and message persistence strategies.
Distributed MessagingKafkadata pipeline
0 likes · 14 min read


21CTOSep 14, 2015 · Backend Development
This article explains Apache Kafka's distributed publish‑subscribe design, core components, storage model, ZooKeeper coordination, performance benchmarks against ActiveMQ and RabbitMQ, and provides Java producer and consumer code examples for building high‑throughput messaging applications.
Apache KafkaDistributed MessagingMessage Queue
0 likes · 16 min read
